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FOREWORD 


The  research  discussed  in  this  report  was  performed  by  the 
Simulation  Team  of  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI)  Rotary-Wing  Aviation 
Research  Unit  at  Fort  Rucker,  Alabama.  ARI  is  committed  to 
enhancing  aviation  training  in  the  Army.  A  cornerstone  of  this 
commitment  is  the  Simulator  Training  Research  Advanced  Testbed 
for  Aviation  (STRATA) .  The  STRATA  training  research  objectives 
are  to  (1)  determine  the  minimal  levels  of  simulator  fidelity 
required  to  meet  specific  task  training  objectives,  (2)  define 
the  most  effective  training  strategies  for  flight  simulator 
technology  and  training  program  design  to  attain  and  sustain 
combat  readiness  for  individual  tasks  and  collective  training, 
and  (3)  delineate  the  effective  ways  to  train  for  new  operational 
equipment,  tactics,  techniques,  and  procedures  based  on  realistic 
simulations  of  battlefield  environments. 

STRATA  has  a  modular  component  design  so  that  it  can  be 
reconfigured  quickly  and  extensively  to  emulate  a  range  of 
training  devices--from  procedures  trainers  to  full  mission 
simulators.  Among  STRATA'S  features  are  (1)  an  automated 
interactive  tactical  environment,  (2)  a  head-  and  eye-tracked 
helmet-mounted  display  providing  for  immersion  in  a  computer¬ 
generated  environment,  and  (3)  the  capability  to  link  to 
Distributed  Interactive  Simulations  (DIS)  as  a  functional  node. 

A  proof  of  concept  demonstration  of  this  networking  capability 
was  performed  on  29  March  1994  when  STRATA  was  linked  to  five 
Apache  simulators  residing  in  Mesa,  Arizona  using  DIS  protocol 
2.03. 


For  the  present  research,  STRATA'S  flexibility  was  exploited 
as  a  prototype  virtual  environment  training  system.  Recently, 
virtual  environment  technologies  have  been  proposed  to  provide 
training-  in  such  critical  military  tasks  as  premission  planning 
and  mission  rehearsal.  This  research  represents  an  attempt  to 
validate  empirically  the  capabilities  of  virtual  environment 
systems  to  provide  training  in  external  terrain,  familiarization. 

This  research  was  presented  to  the  Department  of  Defense 
Training  Technology  Technical  Group  at  their  meeting  in  Palo 
Alto,  California  in  May  1994. 


ZITA  M.  SIMUTIS 
Deputy  Director 
(Science  and  Technology) 
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EDGAR  M.  JOHNSON 
Director 
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USING  VIRTUAL  ENVIRONMENTS  FOR  TERRAIN  FAMILIARIZATION: 
VALIDATION 

EXECUTIVE  SUMMARY _ _ _ 


Requirement: 

To  determine  if  virtual  environment  technology  can  be  used  to 
familiarize  soldiers  with  a  geospecific  location  that  they  have 
never  previously  visited.  Can  important  terrain  information  be 
transmitted  via  this  medium?  Will  this  information  transfer  to 
the  actual,  physical  location? 


Procedure : 

The  design  was  a  two-group,  pretest-posttest  training 
validation  experiment.  The  domain  modeled  in  the  virtual 
environment  was  the  Hanchey  Army  Heliport  (HAH)  located  at  Fort. 
Rucker,  AL.  Both  groups  received  pretests  and  posttests  on  their 
knowledge  of  the  HAH.  Between  pretest  and  posttest  members  of 
both  groups  explored  a  virtual  environment.  The  Hanchey  Group 
performed  90  minutes  of  self-guided  exploration  of  the  virtual 
HAH,  while  the  Control  Group  spent  an  equal  period  of  time 
exploring  a  neutral  virtual  environment  (a  section  of  Arizona) . 


Findings: 

At  pretest  the  groups  did  not  differ  in  their  knowledge  of  the 
Hanchey  Army  Heliport.  At  posttest  the  group  which  had  explored 
the  virtual  Hanchey  knew  significantly  and  substantially  more 
about  the  heliport.  Upon  transfer  to  the  actual  HAH,  members  of 
the  Hanchey  Group  were  able  to  navigate  in  a  minimum  of  time  and 
without  errors  from  location  to  location.  It  was  concluded  that 
virtual  environment  technology  is  a  valid  medium  for  training 
soldiers  in  terrain  familiarization. 


Utilization  of  Findings: 

Research  such  as  this  is  necessary  to  establish  the  validity 
of  virtual  environments  for  training  military  tasks  where  the 
geospecific  location  is  crucial — tasks  such  as  premission 
planning,  mission  rehearsal,  and  special  operations.  These 
results  were  presented  at  the  8th  Department  of  Defense  Training 
Technology  Technical  Group  Meeting  at  Palo  Alto,  California  in 
May,  1994.  Given  the  validation  of  the  basic  concept,  future 
research  will  examine  the  effectiveness  of  virtual  environment 
technology  vis-a-vis  other  training  media. 
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VALIDATION 


Introduction 


Background 

It  is  commonly  acknowledged  in  the  military  that  everything 
except  actual  combat  is  simulation.  Simulation  is  generally 
meant  to  refer  to  one  of  three  classes  of  activity,  either  alone 
or  in  combination  (e.g.,  Drabczuk  &  Tarr,  1993;  Singley,  1993). 
"Live"  simulation  is  concerned  with  real  equipment  in  the  field. 
Field  exercises  at  the  National  Training  Center  with  soldiers  and 
vehicles  instrumented  with  the  Multiple  Integrated  Laser 
Engagement  System  are  an  example  of  this  class  of  simulation. 
"Constructive"  simulation  refers  to  wargames,  models,  and 
analytical  tools  such  as  Janus  and  TACWAR.  These  models 
typically  run  faster  than  real  time  on  mainframe  computers  and 
are  used  to  simulate  large  unit  operations.  "Virtual"  simulation 
refers  to  systems  and  soldiers  in  simulators  fighting  on 
synthetic  battlefields.  One  example  of  this  class  of  simulation 
is  the  Simulation  Networking  (SIMNET)  system  wherein  soldiers  in 
simulators  fight  as  units  in  a  computer-generated  (i.e., 
synthetic)  environment.  Another  example  of  virtual  simulation  is 
aircraft  simulators,  which  have  a  relatively  long  history  in  the 
military  (e.g..  Hays  &  Singer,  1989;  Wiener  &  Nagel,  1988). 

These  three  classes  of  simulation  will  be  linked  in  the 
Army's  current  approach  to  simulation  design  called  Distributed 
Interactive  Simulation  or  DIS.  The  official  definition  of  DIS  is 
reported  by  Drabczuk  and  Tarr  (1993)  as  "DIS  is  a  synthetic 
environment  (at  one  time  described  as  the  'Electronic 
Battlefield')  within  which  humans  may  interact  through 
simulations  at  multiple  sites  networked  using  compliant 
architecture,  modeling,  protocols,  standards  and  data  bases"  (p. 
33).  Another  useful  definition  is  given  by  Bell,  Mastaglio,  and 
Moses  (1993): 

Distributed  Interactive  Simulation  (DIS) :  The  technology  of 
linking  simulators  and  workstations  representing  a  diverse 
set  of  weapons  platforms  and  combat  elements  over  local 
area,  wide  band,  and  long  haul  networks.  Linked  nodes  are 
able  to  operate  within  a  shared  synthetic  environment  and 
experience  common  outcomes  from  combat  events,  (p.  28) 

This  report  discusses  issues  and  research  relevant  to  the  virtual 
simulation  component  of  DIS. 

Virtual  reality  and  virtual  environments.  The  short  history 
of  virtual  reality  and  virtual  environments  has  already  been 
amply  documented  from  a  number  of  different  perspectives  (e.g., 
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Aukstakalnis  &  Blatner,  1992;  Ellis,  1991;  Krueger,  1991; 

Pimentel  &  Teixeira,  1993;  Rheingold,  1991).  There  is,  however, 
a  certain  amount  of  ambiguity  and  overlap  in  the  meaning  of  the 
terms  "virtual  reality"  and  "virtual  environments."  Virtual 
reality  is  sometimes  defined  as  the  experience  of  being  immersed 
in  an  interactive,  three-dimensional,  computer-generated 
environment  (e.g. ,  Pimentel  &  Teixeira,  1993;  U.S.  Army  Research 
Office,  1992) .  Other  times  virtual  reality  is  defined  as  a 
computer-generated,  interactive,  three-dimensional  environment  in 
which  a  person  is  immersed  (e.g.,  Aukstakalnis  &  Blatner,  1992; 
Mogal,  1993;  Pausch,  1993).  The  term  virtual  environment  also 
refers  to  a  computer-generated,  three-dimensional,  interactive 
environment  in  which  a  person  is  immersed  (e.g.,  Ellis,  1991; 
Mowafy  &  Congdon,  1994;  Mowafy  &  Miller,  1993).  The  critical 
defining  features  of  both  terms  are  computer  generation, 
immersion  in  a  three-dimensional  (3— D)  environment,  and 
interaction.  In  this  report  a  virtual  environment  (VE)  is 
defined  as  a  computer-generated,  3-D  environment  in  which  a 
person  is  immersed  and  with  which  the  person  can  interact.  The 
hardware  and  software  which  makes  this  possible  is  called  virtual 
environment  technology  or  VE  technology. 

The  experience  of  immersion  was  described  by  Aukstakalnis 
and  Blatner  (1992)  this  way: 

Being  immersed  means  being  surrounded  by  something; 
everywhere  you  look,  it's  there... To  create  a  sense  of 
immersion  in  a  virtual  environment,  we  must  be  able  to 
surround  ourselves  with  various  stimuli  in  a  manner  that 
makes  sense  and  that  follows  rules  similar  to  those  of  the 
real  world.  That  is,  when  you  turn  your  head  to  the  left, 
you  see  the  objects  to  the  left  of  you.  When  you  walk 
forward,  you  get  closer  to  the  objects  in  front  of  you. 

These  are  elementary  features  of  our  sense  of  being  immersed 
in  an  environment;  and  when  you're  in  a  virtual  environment, 
you  expect  the  same  results,  (p.  27) 

The  typical  mode  of  immersion  is  via  a  head-tracked,  head- 
mounted  (or  helmet-mounted)  display.  Wherever  the  participant 
looks,  the  computer  renders  the  appropriate  view  to  be  seen  in 
real  time  or  near  real  time.  Sometimes,  3-D  sound  is  provided 
through  earphones.  Sound  appears  external  to  the  participant  and 
appears  to  move  with  movements  of  the  participant  or  of  the 
virtual  sound  source.  The  participant  can  interact  with  the 
virtual  environment.  Interaction  may  be  limited  to  locomotion 
through  the  environment  (e.g.,  by  joystick  or  treadmill)  or  may 
include  locomotion  plus  interaction  with  virtual  objects  (e.g., 
push  virtual  buttons,  grasp  and  move  virtual  objects,  doors  open 
upon  proximity,  etc.). 
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Virtual  environments  in  aviation  simulation.  Aviatipn 
simulation  and  training  technologies  are  generally  recognized  as 
key  intellectual  and  technical  precursors  to  VE  technologies. 

The  contributions  of  these  earlier  aerospace  developments  to 
virtual  environments  have  been  discussed  by  many  authors  (e.g., 
Aukstakalnis  &  Blatner,  1992;  Ellis,  1991;  Moshell,  1993; 

Pimentel  &  Teixeira,  1993;  Rheingold,  1991).  Critical 
technologies  such  as  computer-generated  imagery,  immersive 
widescreen  and  helmet— mounted  displays,  and  motion  cueing  were 
developed  for  and  continue  to  be  used  in  commercial  and  military 
aviation  simulation  and  training.  The  first  virtual  environment 
system  was  developed  in  1982  by  U.S.  Air  Force  scientists  and 
called  the  Visually  Coupled  Airborne  Systems  Simulator  or  VCASS 
(cf.  Pimentel  &  Teixeira,  1993;  Rheingold,  1991).  VCASS  was  a 
prototype  VE  cockpit  for  fighter  aircraft.  It  included  a 
computer-generated  environment  based  on  Department  of  Defense 
digitized  terrain  databases  presented  via  an  immersive,  head- 
tracked,  stereoscopic,  fiber-optic,  helmet-mounted  display. 
Targets  could  be  acquired  by  eye-tracking  technology  and  engaged 
by  voice  command  (computer  speech  recognition)  or  virtual  trigger 
pull  (tactile  glove) .  Since  1986  scientists  at  the  National 
Aeronautics  and  Space  Administration  (NASA)  Ames  Research  Center 
have  been  working  on  the  techniques  and  technology  for  real  time 
processing  of  virtual  3— D  acoustic  displays  (Pimentel  &  Teixeira, 
1993;  Wenzel,  1992).  The  Convolvotron  system,  now  widely  used  in 
VE  laboratories  worldwide,  was  developed  as  a  direct  result  of 
this  research  program. 

Besides  these  purely  technical  contributions,  aviation 
simulations  themselves  can  be  considered  a  particular  subset  of 
VE  systems  (cf .  Baum,  1992) .  Modern,  sophisticated,  full-mission 
aviation  simulators  immerse  their  pilots  in  a  computer-generated 
environment  with  which  they  interact  as  they  would  when  piloting 
the  actual,  physical  aircraft.  The  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  (ARI)  owns  and  operates  such  a 
virtual  environment,  full-mission,  helicopter  simulator.  This 
Simulator  Training  Research  Advanced  Testbed  for  Aviation 
(STRATA)  is  described  in  the  Method  section.  It  utilizes  a 
helmet-mounted  display  and  a  fully  functional  cockpit  to  immerse 
pilots  in  a  computer-generated  environment  where  they  can  fly, 
fight,  and  communicate  just  as  they  would  when  performing  an 
actual  mission. 

Virtual  environments  and  training.  There  are  many 
capabilities  inherent  to  VE  technology  which  may  prove  valuable 
for  training.  One  of  these  capabilities  is  the  VE  itself;  Any 
environment  that  can  be  programmed  into  a  database  is  capable  of 
being  visualized  by  the  participant.  Real  world  constraints  do 
not  need  to  apply  to  these  created  worlds.  For  example,  virtual 
environments  exist  for  the  visualization  of  objects  such  as 
molecules  which  are  normally  too  small  to  be  seen.  Virtual 
environments  exist  for  the  visualization  of  objects  such  as 


3 


galaxies  which  are  normally  too  large,  too  dim,  and  too  far  away 
to  be  seen.  Virtual  environments  also  exist  to  allow  the 
participant  to  view  a  new  kitchen  configuration  with  new 
appliances  prior  to  remodeling  or  to  view  and  walk  through  an 
entirely  new  building  before  it  has  been  built.  Virtual 
environments  even  exist  which  allow  for  the  3— D  visualization  of 
data  which  are  normally  seen  if  at  all  in  2-D  tables  or  figures — 
data  such  as  stock  market  statistics  or  weather  measurements. 
Descriptions  of  these  and  other  applications  of  VE  technology  can 
be  found  in  a  number  of  sources  (e.g.,  Aukstakalnis  &  Blatner, 
1992;  Ellis,  1991;  Krueger,  1991;  Pimentel  &  Teixeira,  1993; 
Rheingold,  1991) . 

An  application  of  this  database  capability  for  military 
training  may  be  to  use  VE  technology  to  familiarize  participants 
with  a  location  prior  to  their  going  to  the  actual,  physical 
location  in  person.  Thus,  virtual  environments  have  been 
suggested  for  use  in  premission  planning  or  mission  rehearsal 
(e.g.,  Bell,  Mastaglio,  &  Moses,  1993;  Landry,  1994;  Moshell, 
Blau,  &  Dunn-Roberts ,  1993;  Yuhas,  1993).  A  likely  military 
mission  for  which  VE  training  may  be  a  useful  form  of  mission 
rehearsal  would  be  hostage  rescue  from  a  known  building  by 
special  operations  forces.  VE  technology  could  be  used  to 
familiarize  soldiers  with  the  interior  of  the  building  prior  to 
the  mission.  Research  relating  to  this  mission  has  been 
conducted  by  Witmer,  Bailey,  and  Knerr  (in  preparation).  The 
task  was  to  learn  a  complex  route  through  a  building. 

Participants  were  trained  in  one  of  three  groups  (i.e.,  actual 
building,  VE  building,  and  symbolic)  and  then  transferred  to  the 
actual  building.  Participants  trained  in  the  actual  building 
performed  best  on  the  transfer  test,  followed  by  participants 
trained  in  the  virtual  building.  The  participants  trained  using 
symbolic  techniques  (i.e.,  written  directions,  photographs) 
performed  the  worst  on  the  transfer  test.  This  experiment 
provided  evidence  that  participants  were  able  to  learn  interior 
route  information  using  VE  technology  and  transfer  this  knowledge 
to  the  actual  building. 

Another  capability  of  VE  that  may  prove  to  have  training 
value  is  the  independent  control  of  the  observer's  viewpoint 
(also  called  eyepoint) .  Many  VE  systems  allow  the  participant  to 
adopt  any  viewpoint  he  or  she  chooses  within  the  database  by 
means  of  a  joystick  or  other  controlling  device.  This  means  that 
the  view  the  participant  receives  is  not  limited  to  the 
exigencies  of  the  real  world.  Participants  are  free  to  travel  to 
the  top  of  a  mountain  that  they  could  never  in  reality  climb,  to 
stop  in  mid  air  without  falling,  or  to  fly  right  up  to  an  enemy 
position  without  being  killed.  Moshell  (Moshell,  1993;  Moshell, 
Blau,  &  Dunn-Roberts,  1993)  discusses  the  use  of  "stealth" 
platforms  in  the  SIMNET  VE.  Observers  during  a  virtual  replay  of 
the  Gulf  War  battle  of  73  Easting  control  invisible  eyepoints 
(stealth  platforms)  with  which  they  can  adopt  any  viewpoint  on 


4 


the  battlefield.  They  can  take  a  bird's  eye  view  and  watch  the 
battle  unfold  from  above.  They  can  observe  the  battle  from  the 
friendly  (or  hostile)  commander's  position.  They  can  even  hook 
their  viewpoint  to  one  of  the  players,  either  friendly  or 
hostile,  and  tag  along  wherever  that  player  goes.  This  stealth 
capability  as  it  currently  exists  in  the  Close  Combat  Test  Bed  at 
Fort  Knox  is  described  by  Atwood,  Winsch,  Quinkert,  and  Heiden 
(1994) .  Cosby  (1993)  speculates  how  this  capability  may  be  used 
after  the  turn  of  the  century  if  present  trends  continue. 

Mowafy  and  Miller  (1993)  describe  a  training  program  called 
the  Virtual  Environment  Debrief  Interface  which  employs  this 
capability  of  variable  viewpoint  to  provide  feedback  to  fighter 
pilots  after  flying  a  simulated  air  intercept  mission.  The 
virtual  environment  contains  the  ownship,  the  target,  and,  their 
respective  flight  trajectories  portrayed  as  color-coded 
streamers.  Pilots  can  view  the  air  intercept  replay  record  from 
the  cockpit  of  their  ownship,  the  cockpit  of  the  target,  or  from 
an  external  viewpoint.  Participants  can  move  around  within  the 
VE  and  view  their  performance  from  many  external  perspectives. 
Models  of  expert  fighter  pilots  performing  the  identical  air 
intercept  scenarios  are  also  available  to  be  viewed.  The  purpose 
of  this  training  program  is  to  train  pilots  in  the  ability  to 
visualize  the  geometry  of  an  air  intercept  while  the  target 
aircraft  is  still  beyond  visual  range. 

A  related,  potential  advantage  of  VE  technology  for  training 
is  the  inherent  three  dimensionality  of  the  display  medium. 
Virtual  environments  offer  the  possibility  of  being  able  to  train 
3-D  tasks  directly  using  a  3-D  medium — rather  than  using  2-D 
drawings,  slides,  or  standard  computer  displays.  Mowafy  and 
colleagues  (Mowafy  &  Congdon,  1994;  Mowafy  &  Miller,  1993;  Mowafy 
&  Thurman,  1993)  are  exploring  this  capability  in  their  training 
programs  at  the  Air  Force  Armstrong  Laboratory.  They  are  using 
VE  technology  to  provide  trainees  with  an  opportunity  to  learn 
the  spatial  relationships  of  their  air  combat  tasks  directly 
rather  than  through  2-D  transformations  of  3-D  events.  The  3-D 
Virtual  Environment  Debrief  Interface  (VirDI)  has  been  described 
above  (Mowafy  &  Miller,  1993) .  An  initial  experiment 
investigating  the  training  effectiveness  of  the  VirDI  was  carried 
out  by  Mowafy  and  Thurman  (1993).  The  VirDI  was  compared  to  two 
other  feedback  display  conditions  and  to  a  no  feedback  control 
condition.  The  authors  reported  a  small  advantage  in  air 
intercept  performance  favoring  the  VirDI.  Mowafy  and  Congdon 
(1994)  describe  the  Virtual  Environment  Ground  Command/Control 
Training  Center.  This  VE  is  used  to  train  ground  control 
intercept  operators  in  the  tasks  necessary  to  maintain 
surveillance  over  the  airspace  observed  by  ground-based  and 
airborne  radar  systems.  Participants  are  able  to  move  throughout 
this  3-D  VE,  monitor  the  activities  of  all  aircraft  in  real  time, 
and  perform  basic  ground  control  intercept  operator  tasks. 
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Unlike  life  in  the  real  world,  in  a  virtual  environment 
everything  is  known.  Everything  that  is  seen  or  available  to  be 
seen  is  part  of  the  database.  In  a  military  application,  for 
example,  all  platforms  are  part  of  the  database.  They  are  all 
known  as  to  type,  identity,  armaments,  location,  heading,  range, 
velocity,  and  status  at  all  points  in  time.  Since  the  entire 
virtual  world  and  all  events  that  occur  in  it  are  composed  of 
data,  some  subset  of  these  data  can  be  selected,  recorded, 
analyzed,  and  presented  to  the  participants  in  the  form  of 
feedback.  This  is  another  potential  advantage  of  VE  for 
training.  The  critical  issues  are  what  data  would  be  valuable 
for  training  feedback  and  how  best  to  present  them. 

The  SIMNET  Unit  Performance  Assessment  System  (UPAS;  Meliza, 
Tan,  White,  Gross,  &  McMeel,  1992;  Meliza  &  Tan,  in  preparation) 
is  an  example  of  such  a  feedback  system  for  the  SIMNET  VE.  UPAS 
is  a  personal  computer-based  system  for  use  in  collecting, 
analyzing,  and  presenting  SIMNET  data  to  units  during  after¬ 
action  reviews.  UPAS  collects  network  data  on  vehicle  status  and 
fire  events  during  exercises  and  loads  these  data  into  a 
relational  database  at  the  end  of  the  exercise.  UPAS  contains 
menus  of  graphical  and  tabular  options  for  the  analysis  and 
presentation  of  performance  data.  UPAS  is  designed  to  be  used  in 
addition  to  the  SIMNET  replay  capabilities  to  aid  the  trainer  in 
illustrating  key  events  that  lead  to  exercise  outcomes. 
Instructional  features  for  feedback  also  exist  in  a  wide  variety 
of  nonimmersive  simulation  systems  (cf .  Hays  &  Singer,  1989) . 

The  usefulness  of  VE  technology  for  training  may  be 
dependent  upon  the  task  to  be  trained.  It  has  already  been 
stated  above  with  regard  to  the  work  of  Mowafy  and  colleagues 
that  VE  technology  may  be  particularly  suited  to  training  3-D, 
spatial  tasks.  Regian,  Shebilske,  and  Monk  (1992)  suggest  that 
VE  technology  may  hold  promise  for  the  training  of  visual-spatial 
tasks  because  the  VE  interface  preserves  the  visual-spatial 
characteristics  of  the  simulated  world.  Participants  in  the 
Regian  et  al.  experiment  were  able  to  use  VE  technology  to  learn 
the  spatial  configuration  of  a  virtual  maze.  Unfortunately, 
however,  this  experiment  did  not  include  a  transfer  test  in  an 
identical  real  world  maze.  So  the  implications  of  these  results 
for  training  are  not  clear. 

Kozak,  Hancock,  Arthur,  and  Chrysler  (1993)  found  that 
training  performed  in  a  virtual  environment  did  not  transfer  to  a 
real  world  perceptual -motor  task.  There  were  three  groups  in 
this  transfer  of  training  experiment — no  training,  VE  training, 
and  real  world  training.  All  three  groups  were  transferred  to 
the  real  world  perceptual -motor  task.  The  measure  of  performance 
was  time  to  perform  the  task.  The  group  which  received  training 
on  the  same  real  world  task  performed  significantly  better  than 
the  other  two  groups.  The  VE  trained  group  did  not  differ  from 
the  group  that  received  no  training  at  all. 
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Presence  in  virtual  environments.  The  sense  of  being 
physically  present  within  a  computer-generated  space — the  feeling 
of  actually  "being  there"  when  one  is  immersed  in  a  virtual 
environment — is  called  "presence."  What  are  the  necessary  and 
sufficient  conditions  required  to  create  this  experience  of 
presence?  The  popular  scientific  literature  is  replete  with 
speculation  as  to  the  conditions  which  cause  presence  (e.g., 
Aukstakalnis  &  Blatner,  1992;  Pimentel  &  Teixeira,  1993; 
Rheingold,  1991).  A  listing  of  these  conditions  includes:  rapid 
update  rate,  large  field  of  view,  stereoscopic  display,  high 
resolution  display,  eye  tracking,  head  tracking,  head-mounted 
display,  3-D  sound,  engaging  imagery,  high  image  complexity,  and 
interactivity . 

The  professional  scientific  community  has  recognized  that 
before  definitive  empirical  research  can  be  performed  to 
determine  the  parameters  which  produce  presence,  a  useful  measure 
of  presence  must  first  be  devised  (e.g.,  Held  &  Durlach,  1992; 
Sheridan,  1992) .  Responding  to  this  need,  two  independent  groups 
of  ARI  scientists  have  developed  instruments  designed 
specifically  to  measure  the  reported  experience  of  presence  in 
virtual  environments.  These  are  the  instruments  developed  by 
Psotka  and  Davison  (1993)  and  by  Witmer  and  Singer  (1994).  Given 
valid  and  reliable  measurement  instruments,  scientific  research 
to  explicate  the  conditions  leading  to  the  experience  of  presence 
will  surely  increase. 

Research  Issues 

Military  issue.  The  primary  purpose  of  this  experiment  was 
to  determine  if  virtual  environment  technology  can  be  used  to 
familiarize  soldiers  with  a  geospecific  location  that  they  have 
never  previously  visited.  Can  important  terrain  information  be 
transmitted  via  this  medium?  Will  this  information  transfer  to 
the  actual,  physical  location?  If  VE  technology  is  to  be  used 
for  such  critical  military  tasks  as  premission  planning  and 
mission  rehearsal  then  the  answer  to  the  above  questions  must  be 
yes. 


Technology  validation  issue.  Logically,  the  first  issue  to 
be  addressed  when  developing  a  new  technology  for  training  is 
validation.  Will  instruction  provided  by  this  medium  produce  an 
improvement  in  performance?  The  minimum  research  design  to 
address  this  issue  requires  two  matched  groups — a  trainee  group 
and  a  control  group.  Both  groups  must  be  tested  before 
instruction  (pretest)  to  guarantee  that  they  are  equally  naive  to 
the  subject  matter  prior  to  instruction.  The  two  groups  should 
not  differ  at  pretest.  Both  groups  must  again  be  tested  after 
instruction  (posttest) .  The  trainee  group  should  show  a 
statistically  significant  improvement  in  performance  from  pretest 
to  posttest.  The  control  group,  which  did  not  receive  the 
instruction,  should  not  show  a  statistically  significant 
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improvement  from  pretest  to  posttest.  More  importantly,  at 
posttest  the  performance  of  the  trainee  group  should  be 
significantly  better  than  that  of  the  control  group.  This  issue 
is  being  investigated  in  the  current  experiment. 

Once  a  training  technology  has  been  shown  to  be  valid 
according  to  the  strict  standards  described  above,  a  second  issue 
arises.  This  is  the  issue  of  the  relative  effectiveness  of  the 
new  technology  vis-a-vis  other  technologies.  This  second  issue 
will  be  the  subject  of  future  research. 

Domain.  This  research  required  soldiers  to  use  VE 
technology  for  terrain  familiarization  training.  This  meant 
choosing  some  physical  location  to  be  modeled  in  the  virtual 
environment.  The  domain  chosen  for  virtual  familiarization  was 
the  Hanchey  Army  Heliport  (HAH)  located  on  Fort  Rucker.  This 
heliport  was  chosen  for  a  number  of  reasons.  First,  one 
potentially  valuable  application  of  VE  technology  to  Army 
Aviation  is  to  familiarize  pilots  with  the  physical  features  and 
flight  pattern  information  of  an  airport  prior  to  their  arriving 
there  in  an  aircraft.  Second,  since  the  HAH  is  located  at  Fort 
Rucker  it  is  easily  accessible  for  tests  of  transfer.  Third,  the 
HAH  is  a  basing  field  for  the  AH-64A  Apache  helicopter.  ARI's 
STRATA  facility  at  Fort  Rucker  is  currently  configured  as  an 
Apache  helicopter.  Hence,  the  HAH  virtual  model  could  have  other 
research  uses  in  the  future. 

Instructional  strategy.  The  instructional  strategy  used  for 
familiarization  training  in  this  research  was  self-guided 
exploration.  Soldiers  were  free  to  travel  about  wherever  they 
chose  in  the  VE  and  to  use  their  own  techniques  to  become 
familiar  with  the  physical  features  and  flight  pattern 
information  present  there.  This  strategy  was  chosen  for  two 
reasons.  First,  it  was  an  anchor  point  along  the  dimension  that 
varied  from  structured,  lockstep,  group-based  instruction  to 
unstructured,  self-guided,  individual -based  instruction.  Second, 
both  past  VE  training  (e.g. ,  SIMNET;  Alluisi,  1991)  and  projected 
VE  training  (e.g.,  DIS;  Vaden,  1993)  have  emphasized  free-play 
learning. 

Two  group,  pretest-nosttest ,  experimental  design.  Two 
independent  groups  of  experimentally  naive  soldiers  were  run — the 
Hanchey  Group  and  the  Control  Group.  Members  of  both  groups  were 
pretested  on  their  knowledge  of  the  HAH,  then  allowed  self-guided 
exploration  of  the  virtual  environment,  then  posttested  on  their 
knowledge  of  the  HAH.  At  the  end  of  the  posttest  phase  all 
members  of  both  groups  rated  their  experience  of  presence  in  the 
VE  on  a  questionnaire.  The  independent  variable  was  the 
information  available  in  the  VE.  The  environment  explored  by  the 
Hanchey  Group  was  an  accurate  model  of  the  HAH — which  was  being 
tested.  The  environment  explored  by  the  Control  Group  was  a 
model  of  a  representative  portion  of  the  state  of  Arizona — which 
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was  neutral  for  the  test. 


Hypotheses .  Knowledge  of  the  Hanchey  Army  Heliport:  The 
two  groups  were  not  expected  to  differ  significantly  in  their 
knowledge  of  Hanchey  at  pretest.  The  Hanchey  Group  was  expected 
to  show  significantly  more  knowledge  of  the  HAH  at  posttest  than 
the  Control  Group.  The  Hanchey  Group  was  expected  to  improve 
significantly  from  pretest  to  posttest.  The  Control  Group  was 
expected  not  to  improve  significantly  from  pretest  to  posttest. 
Presence:  The  two  groups  were  not  expected  to  differ 

significantly  in  their  rated  presence  as  a  function  of  which 
virtual  environment  they  explored. 

STRATA.  The  experiment  was  conducted  using  ARI's  STRATA. 
STRATA  is  a  sophisticated  research  simulation  facility  designed 
to  address  issues  pertaining  to  simulator  training  effectiveness 
and  the  training  system  complexity  needed  to  accomplish  specific 
training  objectives.  It  is  modular  and  can  be  reconfigured  to 
represent  different  training  devices  with  different  visual, 
motion,  cockpit,  and  aeromodel  subsystems.  For  a  detailed 
description  of  STRATA  see  Kurts  and  Gainer  (1991).  For  a. 
description  of  research  conducted  to  validate  STRATA  see  Stewart 
(1994).  A  recent  report  by  Stewart,  Wightman,  and  Gainer  (1993) 
discusses  future  research  planned  to  be  conducted  in  STRATA.  The 
immediate  objective  of  the  research  program  is  to  employ  STRATA 
to  address  four  major  issues:  (a)  the  minimal  level  of  fidelity 
required  to  meet  training  objectives;  (b)  the  most  effective  (in 
terms  of  outcome  and  cost)  use  of  flight  simulation  technology  to 
attain  and  sustain  combat  readiness;  (c)  the  most  effective  ways 
of  defining  the  use  of  new  operational  equipment,  tactics, 
techniques,  and  procedures  in  a  realistic  threat  environment;  and 
(d)  incorporate  lessons  learned  through  STRATA  into  the 
development  of  modular,  portable  simulation  systems. 


Method 


Participants 

Twelve  participants  were  randomly  assigned  either  to  the 
Hanchey  Group  or  to  the  Control  Group.  Six  were  assigned  to  each 
group.  All  participants  were  soldiers  from  aviation  units  at 
Fort  Rucker.  All  were  volunteers  and  all  signed  the  Volunteer 
Agreement  Affidavit  (DA  Form  5303-R,  May  88).  Criteria  for 
selection  to  this  experiment  were  that  participants  be  soldiers 
from  Fort  Rucker  who  had  never  visited  the  Hanchey  Army  Heliport. 
Gender,  rank,  and  military  occupational  specialty  were  irrelevant 
for  this  experiment  and  were  not  included  as  criteria  for 
selection.  Table  1  presents  a  demographic  summary  of  the 
participants  in  this  experiment. 
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Table  1 


Demographic  Description  of  Experimental  Participants* 


Aviator 
Yes:  7 

No:  5 


Gender 

Rank 

Male: 

10 

Officers:  4 

Female: 

2 

CPT  (2) 

1LT  (1) 

Acre  fvears) 

2LT  (1) 

Mean: 

28.33 

Warrant  Officers:  5 

Median: 

27 

CW2  (1) 

Range : 

23  -  39 

W01  (4) 

Enlisted: 
CPL  (1) 
PFC  (1) 
PVT  (1) 


*  N  =  12 


Apparatus  and  Materials 

A  general  description  of  STRATA.  STRATA  is  currently 
configured  as  an  AH-64A  Apache  helicopter.  There  are  two 
separated  cockpits,  pilot  and  copilot-gunner,  constructed  from 
salvaged  AH-64A  cockpits.  Both  cockpits  contain  fully  functional 
and  integrated  flight  instruments,  sensors,  displays,  and  mission 
packages.  Flight  controls  are  linked  between  the  cockpits. 

Flight  controls  produce  accurate  force  feedback.  The  simulator 
flies  as  an  Apache  due  to  accurate  aerodynamic  modeling.  Motion 
cueing  is  provided  to  both  cockpits  by  means  of  hydraulically 
actuated  pneumatic  G-seats.  The  cockpits  communicate  via 
intercom.  Both  cockpits  are  continuously  ventilated  with  cooled 
air. 


Imagery  for  both  out-the-window  views  as  well  as  all  sensor 
displays  is  provided  by  an  Evans  and  Sutherland  ESIG-1000  image 
generator.  This  image  generator  uses  eleven  channels  distributed 
over  three  eyepoints  (pilot,  copilot-gunner,  and  sensor) .  There 
are  two  infrared  post  processors  for  both  Apache  forward  looking 
infrared  radiation  (FLIR)  sensor  displays.  Image  update  rate  is 
60  Hz. 

In  the  pilot  cockpit  out-the-window  scenes  are  presented  via 
a  fiber  optic  helmet-mounted  display  (FOHMD) .  This  display 
presents  four  channels  of  visual  information — left  and  right 
background  and  left  and  right  inset  (see  Figure  1) .  The 
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Individual  Background  and  Inset  Fields 


Combined  Background  and  Inset  Fields 


Figure  1.  Schematic  diagram  of  the  background  and  inset  fields  of  the 
fiber  optic  helmet-mounted  display.  (The  relative  scale  among  the 
objects  is  accurate.) 
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instantaneous  background  field  of  view  (FOV)  is  127  degrees 
horizontal  and  66  degrees  vertical.  However,  since  the  helmet 
employs  an  infrared  head  tracking  system  and  presents  imagery 
wherever  the  pilot  looks,  the  effective  field  of  regard  is  360 
degrees.  Infrared  eye  tracking  positions  the  left  and  right  high 
resolution  insets  at  the  center  of  the  viewer's  gaze.  These 
insets  subtend  24  degrees  horizontal  and  18  degrees  vertical. 
Resolution  of  the  background  displays  is  5.0  arcminutes  while 
resolution  of  the  insets  is  1.5  arcminutes.  Each  left  and  right 
background  displays  512  lines,  524,000  pixels,  and  1200  polygons. 
Each  inset  displays  1024  lines,  1,048,000  pixels,  and  1200 
polygons.  Luminance  is  greater  than  35  footlamberts.  Contrast 
ratio  is  50  to  1.  The  FOHMD  weighs  five  pounds  but  part  of  this 
weight  is  supported  by  three  wires  which,  though  attached  to  the 
structure  of  the  simulator,  allow  full  freedom  of  movement.  (The 
standard  Apache  flight  helmet,  the  Integrated  Helmet  and  Display 
Sight  Sub-system  or  IHADSS,  also  weighs  five  pounds.)  A  helmet 
is  custom  fitted  and  optically  calibrated  to  each  participant. 

All  control  of  the  simulator  is  exercised  from  the 
Experimenter-Operator  Station  (EOS).  An  experimenter  can 
initiate  simulator  scenarios,  monitor  participants  in  their 
cockpits,  communicate  with  participants  over  the  intercom,  and 
observe  via  repeaters  the  visual  scene  presented  to  either 
cockpit.  The  Interactive  Tactical  Environment  Management  System 
is  used  to  create  scenarios,  control  multiple  intelligent 
synthetic  players  (both  friendly  and  hostile) ,  control  weapons, 
terrain,  and  weather.  The  Blue-Red  Team  Station  allows  the 
experimenter  to  control  any  player  in  the  scenario  from  the  EOS. 
There  is  also  a  Database  Management  System  and  Data  Recording  and 
Analysis  Station,  which  support  tactical  scenario  generation  and 
performance  measurement.  A  visual  database  modeling  workstation 
can  be  used  to  modify  existing  visual  databases  and  create  new 
ones. 


Experiment-specific  features  of  STRATA.  The  pilot  cockpit 
was  used  in  this  experiment.  No  flight  instruments  or  flight 
controls  were  used.  They  were  covered  by  a  black  blanket. 
Cockpit  lights  and  power  were  turned  off.  The  G-seat  was  not 
used.  No  vestibular  or  proprioceptive  cues  to  motion  were 
present  in  this  experiment.  Ventilation  remained  on. 

Two  joysticks  were  attached  to  the  seat  at  a  comfortable 
armchair  height — one  on  the  left  and  one  on  the  right.  The  left 
joystick  controlled  up  and  down  movement.  The  right  joystick 
controlled  forward  and  backward  movement  as  well  as  left  and 
right  turns.  A  button  on  the  right  joystick  was  the  reset 
button.  Pushing  this  button  caused  the  image  generator  to 
reposition  the  participant  to  a  particular  location  in  each 
virtual  environment. 
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The  fully- immersive,  stereoscopic  FOHMD  was  used  for  the 
presentation  of  all  visual  information  while  in  the  virtual 
environment.  Head  tracking  was  enabled.  The  combination  of  the 
infrared  head-tracking  system,  the  rate  sensing  hardware  mounted 
on  the  helmet,  the  prediction  algorithms,  and  a  60  Hz  update  rate 
produced  a  virtual  experience  with  no  perceptible  head  tracking 
delay.  Eye  tracking  was  not  used  in  this  experiment.  Both  the 
left  and  the  right  high  resolution  insets  were  fixed  forward. in 
the  center  of  the  visual  field.  A  helmet  was  individually  fitted 
and  optically  aligned  for  each  participant. 

During  the  experiment  a  large  black  curtain  was  drawn 
completely  around  the  cockpit.  This  served  to  prevent  ambient 
light  from  the  dimly  lit  simulator  bay  from  reaching  the  cockpit. 
Once  seated  in  the  cockpit  the  only  light  was  that  of  the  virtual 
scene  displayed  by  the  FOHMD.  Participants  were  completely 
immersed  in  the  VE  and  could  view  the  environment  by  looking  in 
any  direction.  Participants  were  not  a  disembodied  eyepoint, 
however.  When  looking  down  participants  could  see  the  black 
virtual  carpet  on  which  they  were  seated.  This  virtual  carpet 
covered  the  space  immediately  under  their  chair  and  feet  being 
2.5  feet  wide  by  3.0  feet  long.  It  was  this  virtual  carpet  which 
could  be  made  to  fly  throughout  the  VE  under  participant  control 
by  manipulation  of  the  left  and  right  joysticks. 

During  the  experiment  the  four  channels  of  the  FOHMD  (left 
and  right  background,  left  and  right  insets)  were  continuously 
monitored  by  the  experimenter  over  four  repeater  displays  at  the 
EOS.  The  intercom  channel  was  also  continuously  monitored  by  the 
experimenter  over  earphones  at  the  EOS . 

Virtual  environments — Hanchev  Army  Heliport  and  Arizona. 

Two  virtual  environments  were  used  in  this  experiment — the 
Hanchey  Army  Heliport  environment  and  a  section  of  the  Arizona 
environment  near  the  town  of  Mesa.  The  HAH  environment  was 
created  for  this  and  related  research.  The  Arizona  environment 
used  in  this  research  was  one  terrain  module  taken  from  the 
Arizona  database.  The  entire  state  of  Arizona  is  available  in 
this  database  and  is  the  baseline  database  used  in  STRATA. 

The  Hanchey  Army  Heliport  virtual  environment  (HAH-VE)  is  an 
accurate,  fullscale  representation  of  the  actual,  physical  HAH 
located  on  Fort  Rucker.  The  HAH-VE  measures  0.72  miles  in  east- 
west  orientation  and  0.52  miles  in  north-south  orientation.  The 
HAH-VE  is  located  in  the  center  of  a  flat,  green,  terrain  square 
measuring  40.34  miles  on  a  side.  Nothing  is  visible  in  the  green 
expanse  beyond  HAH-VE. 

Priority  for  inclusion  in  the  HAH-VE  went  to  large, 
permanent  or  semipermanent,  exterior  features  which  identify  HAH 
and  to  features  which  are  relevant  to  the  flight  training  mission 
of  the  heliport.  That  is,  features  were  included  if  they  were 
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exterior,  large,  relatively  permanent,  distinctive,  or  important 
to  the  flight  mission  of  the  heliport.  If  features  were  judged 
to  be  critical  either  for  identification,  navigation,  or  flight 
safety  they  were  included. 

Considerable  time  and  effort  were  expended  to  acquire  the 
basic  physical  and  flight  pattern  knowledge  required  to  model 
Hanchey.  Multiple  visits  to  HAH  were  undertaken  by  the  database 
modeling  and  research  staff.  Subject  matter  experts  were 
consulted.  Defense  Mapping  Agency  data  were  acquired.  Maps  were 
analyzed.  U.S.  Army  Aviation  Center  (USAAVNC)  flight  regulations 
were  examined.  Measurements  were  made  of  all  permanent  and 
semipermanent  structures  at  HAH.  Videotape  and  photographic 
records  were  made  of  the  entire  heliport  and  all  structures  both 
from  the  ground  and  from  the  air. 

A  two-dimensional,  bird's  eye  view  of  the  largest  physical 
features  incorporated  in  the  HAH-VE  is  presented  in  Figure  2. 
Objects  presented  in  this  diagram  are  drawn  to  scale.  This 
figure  is  presented  with  north  at  the  top.  The  HAH  is  roughly 
H T"  shaped.  The  long  axis  of  HAH  is  oriented  generally  east-west 
and  is  presented  left-right  in  the  figure.  The  shorter  axis  of 
Hanchey  is  oriented  generally  north-south  and  is  presented  top- 
bottom  in  the  figure.  The  HAH  is  situated  on  a  plateau  with  the 
ground  surrounding  the  flat  tarmac  sloping  gently  away.  This  is 
also  modeled  in  the  virtual  Hanchey  where  the  stem  of  the  "T" 

(the  HAH  "panhandle")  slopes  downhill  to  the  south. 

All  objects  in  the  virtual  Hanchey  environment  were  modeled 
to  actual  size  and  presented  in  their  actual  locations.  Colors 
were  tuned  to  match  the  colors  recorded  in  photographs  and  on 
videotape,  where  feasible.  All  signs  and  logos  were  texture 
mapped  onto  buildings  in  their  correct  positions. 

Among  the  physical  features  modeled  in  the  HAH-VE  were  all 
nineteen  helipads  (including  one  VIP  pad),  all  correctly 
designated,  complete  with  all  aircraft  parking  ramps,  taxi  lanes, 
and  overrun  areas.  All  thirty  permanent  or  semipermanent 
buildings  were  modeled,  including  three  large  hangars,  Cobra 
Hall,  Chinook  Hall,  classroom  buildings,  storage  buildings,  the 
fire  station,  the  operations  building,  the  snack  bar,  and  the 
guard  shack.  Critical  flight  related  structures  were  modeled, 
including  the  control  tower,  the  beacon  tower,  the  antenna  pole, 
all  three  windsocks,  and  all  four  fuel  tanks.  Miscellaneous, 
distinctive  objects  were  also  modeled,  including  two  fire  trucks, 
two  natural  gas  tanks,  one  water  tank,  one  satellite  receiver 
dish,  and  all  paved  automobile  parking  lots.  Some  buildings  had 
large,  distinctive  signs  or  logos  mapped  onto  them.  The  Apache 
hangar  had  two  large  Apache  logos — one  facing  east  and  one  west. 
The  Chinook  hangar  had  a  large  field  elevation  sign  facing  east. 
The  Kiowa  hangar  had  a  large  "Warrior  Country"  logo  and  a  large 
field  elevation  sign  both  facing  south.  Chinook  Hall  had  its 
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Figure  2.  Scale  diagram  of  largest  physical  features  present  in  the  Hanchey  Army  Heliport 
virtual  environment  when  viewed  from  above. 


"Chinook  Hall  Windjammers"  logo  complete  with  CH-47  silhouette 
facing  east.  Cobra  Hall  had  its  large,  distinctive  "Cobra  Hall" 
logo  complete  with  green  cobra  snake  on  a  red  background  facing 
south. 

Included  among  the  physical  features  of  the  HAH  to  be 
modeled  were  the  helicopters  based  there.  There  are  four 
helicopter  types  based  at  Hanchey.  These  are  the  AH-64  Apache, 
the  AH-1  Cobra,  the  OH-58  Kiowa,  and  the  CH-47  Chinook.  The 
Apache  is  based  on  the  west  side  of  HAH,  while  the  other  three 
types  are  based  on  the  east  side.  Dozens  of  these  four 
helicopter  types  are  parked  at  Hanchey  on  any  given  day.  The 
HAH-VE  included  one  exemplar  of  each  of  the  four  helicopter 
types.  All  helicopters  were  parked  on  their  appropriate  side  of 
Hanchey  and  in  their  correct  orientation.  The  Apache  was  parked 
in  row  B  on  the  west  side  of  HAH-VE  in  the  north-south 
orientation.  The  Cobra  was  parked  in  row  F  on  the  east  side  of 
HAH-VE  in  the  east-west  orientation.  The  Kiowa  was  parked  in  row 
E  on  east  HAH-VE  in  the  east-west  orientation.  The  Chinook  was 
parked  in  row  H  on  east  HAH-VE  in  the  east-west  orientation. 

During  the  experiment,  all  four  helicopters  cycled  through 
their  respective  traffic  patterns  in  sequence  continuously.  One 
by  one  each  helicopter  would  start  its  rotor  turning,  move  from 
its  parking  place  to  the  taxi  lane,  taxi  to  its  assigned 
departure  helipad,  pick  up  to  a  hover,  depart  along  its  departure 
lane  and  climb  to  traffic  pattern  altitude,  and  then  fly  the 
crosswind,  downwind,  base,  and  final  approach  legs  of  the  traffic 
pattern.  Each  helicopter  would  then  approach  and  hover  over  its 
assigned  landing  pad,  land,  taxi  to  its  assigned  parking  space, 
park  in  the  correct  orientation,  and  stop  its  rotor.  At  this 
point  another  helicopter's  rotor  would  begin  to  turn  and  it  would 
perform  its  traffic  pattern  flight  sequence.  This  continuous 
cycle  was  performed  in  the  order  Cobra,  Kiowa,  Chinook,  and 
Apache.  It  continued  as  long  as  the  HAH-VE  was  enabled. 

All  phases  of  the  traffic  pattern  from  park  through  flight 
back  to  park  were  carried  out  in  accordance  with  USAAVNC  flight 
regulations.  All  speeds,  altitudes,  distances,  departure  lanes, 
and  approach  lanes  were  accurate  virtual  representations  of  the 
actual  flight  rules  followed  at  the  HAH.  For  example,  the 
traffic  pattern  altitude  for  the  west  Hanchey  pattern  is  500  feet 
mean  sea  level  (MSL)  but  800  feet  MSL  for  the  east  Hanchey 
pattern.  The  HAH  field  elevation  is  311  feet  MSL. 

One  of  the  key  advantages  of  VE  technology  is  the  capability 
to  make  data  visible.  A  salient  virtual  feature  of  the  HAH-VE 
was  not  a  physical  feature  of  the  actual  HAH.  A  large,  red,  3-D 
compass  arrow  was  always  present  in  the  lower  center  field  of 
view.  This  arrow  pointed  to  magnetic  north  and  had  a  white  "N" 
painted  on  it.  This  arrow  always  pointed  north  no  matter  how  the 
participant  turned  the  FOHMD  or  moved  the  virtual  carpet. 
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Further,  there  was  no  perceptible  lag  to  the  directional 
information  provided  by  the  arrow — it  kept  pace  with  all  helmet 
or  carpet  movements  no  matter  how  quickly  made.  Being  positioned 
in  the  lower  center  FOV,  the  arrow  provided  compass  direction 
without  blocking  the  visual  scene. 

Figures  3  through  8  present  various  views  of  the  HAH-VE  as 
photographed  from  a  repeater  display  in  the  EOS.  Visible  in  the 
foreground  of  all  figures  is  the  black,  rectangular,  virtual 
carpet  and  the  north-pointing  compass  arrow.  Figure  3  shows  a 
portion  of  the  OH-58  Kiowa  hangar.  Figure  4  presents _ Cobra  Hall 
and  buildings  to  the  north.  Figure  5  displays  a  portion  of  the 
Apache  hangar  in  the  foreground  with  the  control  tower  in  the 
background.  Figure  6  shows  a  parked  AH-64  Apache  with  fuel  tanks 
in  the  background.  Figure  7  displays  a  parked  CH-47  Chinook. 
Figure  8  presents  administrative,  storage,  and  hangar  buildings 
at  the  north  end  of  the  field. 
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Figure  3.  HAH-VE:  OH-58  Kiowa  hangar. 


Figure  A.  HAH-VE:  Cobra  Hall  and  buildings  to  the  north. 


Figure  5.  HAH-VE:  AH-64  Apache  hangar  in  foreground  with  control  tower 
in  background. 


Figure  6.  HAH-VE:  AH-64  Apache  helicopter  with  fuel  tanks  in  background. 
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Figure  8.  HAH-VE:  Buildings  at  north  end  of  field. 
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Pushing  the  reset  button  while  in  the  HAH-VE  would 
immediately  reposition  the  participant  to  a  location  in  front  of 
the  southwest  section  of  the  OH-58  Kiowa  hangar  facing  north. 

From  this  position  the  participant  could  clearly  see  the  Warrior 
Country  logo  painted  on  the  side  of  the  building.  This  reset 
location — including  the  view  of  the  logo — was  the  same  position 
from  which  the  Pretest  Part  2  and  Posttest  Part  2  Questionnaires 
were  administered  on  the  visits  to  the  actual  HAH  during  the 
experiment  (see  Procedure  below) . 

STRATA  did  not  allow  participants  to  pass  underground  while 
in  the  HAH-VE.  Seated  on  the  virtual  carpet  with  the  left 
joystick  all  the  way  down,  a  participant's  minimum  eyelevel  was 
set  at  two  feet  above  ground  level.  A  participant  had  only  to 
look  down  to  see  whatever  terrain  feature  was  below  this 
eyelevel . 

The  Arizona  virtual  environment  (Ariz-VE)  was  located  one 
million  feet  (189.39  miles)  east  of  the  HAH-VE  in  the  same 
database.  The  two  VEs  were  separated  by  empty  space  and  were,  of 
course,  not  intervisible.  Having  both  the  Hanchey  and  the 
Arizona  environments  in  the  same  database  meant  that  participants 
could  be  "teleported"  from"  one  to  the  other  in  a  matter  of 
seconds  under  experimenter  control  from  the  EOS . 

The  Ariz-VE  was  one  terrain  module  taken  from  STRATA'S 
Arizona  database.  This  module  was  a  square  measuring  10.08  miles 
on  a  side.  It  was  centered  east  of  Phoenix  and  included  part  of 
Mesa,  Arizona.  The  Ariz-VE  contained  urban,  residential,  and 
desert  terrain.  The  three  terrain  types  included  appropriate 
types  and  densities  of  buildings,  businesses,  churches,  houses, 
towers,  playgrounds,  automobiles,  roads,  parking  lots,  signs, 
streams,  and  vegetation.  The  Ariz-VE  did  not  contain  any 
aircraft  or  moving  models. 

The  3-D,  north-pointing  compass  arrow  was  also  present  in 
the  lower  center  FOV  in  the  Ariz-VE.  It  functioned  exactly  the 
same  as  it  did  in  the  HAH-VE.  Pushing  the  reset  button  while  in 
the  Ariz-VE  would  immediately  reposition  the  participant  to  a 
location  in  front  of  a  particular  gasoline  station  in  the 
residential  terrain.  As  in  the  HAH-VE,  STRATA  did  not  allow 
participants  to  pass  underground  while  in  the  Ariz-VE.  Minimum 
eyelevel  was  again  set  at  two  feet  above  ground  level. 

Forms,  questionnaires,  and  other  measures  of  performance. 

All  participants  filled  out  and  signed  the  Volunteer  Agreement 
Affidavit  (DA  Form  5303-R,  May  88) .  Information  in  this  form 
served  to  identify  the  participant,  the  research,  the 
experimenter,  the  agency,  and  to  guarantee  that  all  participants 
knew  they  were  participating  voluntarily  and  could  withdraw  at 
any  time. 
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All  participants  completed  and  signed  the  Demographic 
Information  Form  (see  Appendix) .  This  form  served  to  provide 
information  as  to  name,  social  security  number,  age,  rank,  unit, 
telephone  number,  aviator  status,  and  whether  they  had  ever 
visited  the  HAH. 

All  participants  took  the  Pretest  Part  1  Questionnaire  (see 
Appendix) .  This  test  consisted  of  fourteen  questions  assessing 
knowledge  of  HAH.  Questions  were  concerned  primarily  with  flight 
related  physical  features  of  Hanchey.  Questions  were  also  asked 
about  the  helicopters  based  at  HAH  and  the  flight  traffic 
patterns  of  east  and  west  Hanchey.  Some  questions  required 
participants  to  fill  in  the  correct  answers  and  others  were 
multiple  choice.  Participants  were  required  to  answer  each 
question.  Guessing  was  permitted.  Participants  who  did  not  know 
the  answer  and  did  not  wish  to  guess  were  instructed  to  write 
"DK"  for  "don't  know."  There  were  eighteen  points  possible  on 
this  pretest. 

All  participants  were  administered  the  Pretest  Part  2 
Questionnaire  (see  Appendix) .  This  test  consisted  of  seventeen 
questions  assessing  knowledge  of  HAH.  The  questions  concerned 
the  physical  features  of  Hanchey,  the  helicopters  based  there, 
and  the  traffic  pattern.  Some  questions  covered  the  same 
information  as  in  the  Pretest  Part  1  Questionnaire  but  in  a 
different  format  and  context.  All  questions  were  multiple 
choice.  All  questions  were  asked  verbally  by  the  experimenter 
and  all  answer  options  listed.  Participants  reported  their 
chosen  answers  verbally.  Participants  were  required  to  answer 
all  questions.  Guessing  was  permitted.  A  participant  who  did 
not  know  the  correct  answer  and  did  not  wish  to  guess  could 
respond  "don't  know."  Don't  know  was  always  one  of  the  possible 
answer  options.  There  were  twenty- two  points  possible  on  this 
pretest. 

All  participants  took  the  Posttest  Part  1  Questionnaire  (see 
Appendix) .  This  test  was  identical  to  the  Pretest  Part  1 
Questionnaire  in  every  way  except  its  title.  It  contained  the 
same  fourteen  questions  in  the  same  order.  Content, 
instructions,  and  scoring  were  identical  to  the  description  given 
above  for  the  Pretest  Part  1  Questionnaire.  There  were  eighteen 
points  possible. 

All  participants  filled  out  the  Posttest  Part  1  Object 
Placement  Test  (see  Appendix) .  This  test  assessed  participants' 
knowledge  of  the  physical  features  of  the  HAH.  It  consisted  of  a 
generalized  diagram  of  the  outlines  of  the  HAH  and  a  list  of  key 
objects  to  be  placed  in  their  appropriate  locations  on  the 
diagram.  Objects  to  be  placed  were  identified  with  descriptive, 
uppercase  letters.  Participants  were  not  required  to  draw  actual 
features,  merely  to  place  the  appropriate  descriptive  letters  in 
the  appropriate  locations  in  the  diagram.  This  test  was  designed 
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to  measure  knowledge  of  the  locations  of  key  Hanchey  features, 
not  artistic  ability.  There  were  a  total  of  thirty-four  objects 
to  be  placed  correctly  in  the  diagram.  Guessing  was  permitted. 

If  a  participant  did  not  know  where  to  place  an  object  and  did 
not  wish  to  guess  he  or  she  could  leave  it  blank.  An  object  was 
scored  as  being  correctly  placed  if  any  portion  of  it  was  located 
within  0.25  inch  of  its  correct  location.  There  were  thirty-four 
points  possible  on  this  posttest. 

All  participants  were  administered  the  Posttest  Part  2 
Questionnaire  (see  Appendix) .  This  test  was  identical  to  the 
Pretest  Part  2  Questionnaire  in  every  way  except  its  title.  It 
contained  the  same  seventeen  questions  in  the  same  order. 

Content,  instructions,  and  scoring  were  identical  to  the 
description  given  above  for  the  Pretest  Part  2  Questionnaire. 
There  were  twenty-two  points  possible. 

The  Hanchey  Army  Heliport  Walking  Navigation  Test  was 
administered  only  to  members  of  the  Hanchey  Group  (see  Appendix) . 
This  test  measured  participants'  knowledge  of  the  physical 
features  of  the  HAH  by  asking  them  to  use  their  knowledge  to 
navigate  from  one  location  to  another  in  the  real  world  at  the 
actual  HAH.  This  transfer  test  consisted  of  two  data  collection 
walks.  In  both,  participants  began  at  an  initial  position  and 
walked  to  a  goal  position  while  passing  two  waypoints  in  order 
along  the  route.  Neither  the  goals  nor  the  waypoints  were 
visible  from  the  initial  positions.  Further,  to  get  to  the  goal 
locations  by  passing  the  two  waypoints  required  following  a 
circuitous  route.  In  addition,  neither  the  goal  position  nor  the 
waypoints  for  the  second  walk  were  visible  during  the  first  walk. 

After  informing  the  participant  of  the  goal  and  the 
landmarks  to  pass  along  the  route,  the  experimenter  followed 
directly  behind  the  participant  as  he  or  she  walked.  Three 
measures  of  performance  were  recorded  by  the  experimenter  during 
each  walk:  time,  in  seconds,  to  walk  from  start  to  goal;  number 
of  wrong  turns  taken;  and  the  experimenter's  judgement  of  the 
participant's  confidence  level  based  upon  overt  behavior.  The 
confidence  rating  was  a  three-point  scale  where  1  meant  "unsure, 
looking,  searching,  halting  steps"  and  3  meant  "sure,  confident, 
not  searching,  direct  path."  A  rating  of  2  represented  a  judged 
level  of  confidence  between  1  and  3. 

The  Hanchey  Army  Heliport  Knowledge  Rating  Form  was  filled 
out  only  by  members  of  the  Hanchey  Group  (see  Appendix) .  This 
form  asked  each  participant  to  rate  the  amount  of  knowledge  of 
Hanchey  gained  by  them  in  the  HAH-VE  on  a  five-point  scale.  The 
five  rating  scale  levels  were  (1)  None,  (2)  Some,  (3)  Adequate, 

(4)  Much,  and  (5)  Very  Much.  The  levels  1,  3,  and  5  were 
anchored  with  written  descriptions  of  what  the  rating  meant. 

These  descriptions  were  included  on  the  form  (see  Appendix) .  The 
purpose  of  this  measure  was  to  get  an  index  of  the  amount  that 
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the  Hanchey  Group  participants  themselves  thought  they  had 
learned  in  the  virtual  Hanchey  environment. 

All  participants  filled  out  the  Presence  Questionnaire 
developed  by  Witmer  and  Singer  (1994).  This  questionnaire  is 
presented  in  the  Appendix.  Presence  was  defined  by  Witmer  and 
Singer  as  the  subjective  experience  of  being  in  one  environment 
(there)  when  one  is  physically  in  another  environment  (here) . 

The  experience  of  presence  in  the  VE  was  measured  by  this 
instrument.  The  questionnaire  contained  thirty-two  questions, 
each  of  which  required  marking  an  "X"  along  a  seven-point, 
Likert-type  scale.  Scoring  instructions  for  this  questionnaire 
as  well  as  evidence  of  its  validity  and  reliability  were  provided 
in  Witmer  and  Singer  (1994)  .  Higher  scores  meant  greater 
reported  presence. 


Procedure 


General .  Participation  in  this  experiment  required  soldiers 
to  be  scheduled  for  two  visits  to  ARI  on  each  of  two  separate 
days.  Day  one  activities  included  helmet  fitting  and 
preorientation.  This  took  less  than  an  hour.  Day  two  activities 
were  the  experiment  proper  and  included  experimental  orientation, 
pretest,  exploration  of  the  virtual  environment,  posttest, 
Presence  Questionnaire,  debrief,  and  question  and  answer.  Day 
two  began  when  each  soldier  arrived  at  ARI  at  0700  hours  and 
continued  until  approximately  1230  hours.  One  soldier  was  run  in 
the  experiment  per  day. 

Helmet  fitting  and  preorientation.  Upon  arrival  at  ARI, 
participants  met  with  the  experimenter  (DJ)  who  checked  to  make 
sure  that  they  had  never  visited  the  HAH,  confirmed  the  date  and 
time  for  their  participation  in  the  experiment  proper,  and 
briefed  them  in  general  terms  on  the  nature  of  the  research  and 
what  would  be  required  of  them.  Participants  were  then  fitted 
for  a  helmet  to  support  the  FOHMD  and  had  the  optics  calibrated 
for  their  eyes.  After  fitting  and  calibration  each  helmet  was 
labelled  with  the  name  of  the  participant  and  then  stored  in  a 
recorded  location  for  use  during  the  experiment.  At  no  time 
during  these  activities  were  participants  allowed  to  enter  either 
the  simulator  bay  or  the  EOS. 

Experimental  orientation.  Upon  arrival  at  ARI  on  the  day  of 
the  experiment  participants  were  briefed  on  the  nature  of  the 
research,  their  place  in  the  research,  and  what  was  required  of 
them.  Questions  were  answered,  where  appropriate.  Participants 
were  told  that  there  was  going  to  be  a  pretest  followed  by  self- 
guided  exploration  of  a  VE  and  then  a  posttest.  Participants 
were  not  told  that  there  were  two  groups  (Hanchey,  Control)  and 
were  not  told  their  group  identity.  Then  participants  signed  the 
Volunteer  Agreement  Affidavit.  Participants  then  completed  and 
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signed  the  Demographic  Information  form.  Besides  providing  the 
demographic  information  listed  above,  this  form  once  again  asked 
them  if  they  had  ever  visited  the  HAH.  All  participants 
stipulated  that  they  had  not. 

Pretest.  Immediately  after  the  orientation,  each 
participant  filled  out  the  Pretest  Part  1  Questionnaire  at  ARI. 
Then  each  participant  was  taken  by  the  experimenter  to  the  HAH 
located  on  Fort  Rucker.  The  participant  donned  opaque  goggles, 
provided  by  the  experimenter,  prior  to  getting  within  sight  of 
Hanchey.  Opaque  goggles  were  worn  in  order  to  guarantee 
experimental  control  of  each  participant's  visual  exposure  to 
HAH.  Upon  arrival  at  Hanchey,  the  participant— -still  wearing 
goggles — was  lead  by  the  experimenter  to  a  particular  location 
facing  the  southwest  section  of  the  Kiowa  hangar.  Here  the 
participant  was  instructed  to  stand  still  with  head  and  eyes 
fixed  forward.  With  the  experimenter  positioned  in  front  of  the 
participant,  the  participant  removed  the  goggles.  The 
experimenter  read  the  instructions  for  the  Pretest  Part  2 
Questionnaire  to  the  participant  (see  Appendix)  and  then 
administered  the  questionnaire.  Participants  were  instructed  to 
provide  verbal  responses  to  the  questions  read  to  them  from  the 
response  options  supplied.  The  experimenter  circled  each 
response  on  the  clipboard  answer  sheet.  The  experimenter  also 
monitored  the  participant  to  guarantee  that  he  or  she  remained 
stationary  with  head  and  eyes  pointed  forward  during  this 
pretest. 

After  answering  all  questions,  and  while  remaining 
stationary  with  head  and  eyes  pointed  forward,  the  participant 
again  donned  the  goggles  for  the  trip  back  to  ARI.  Once  out  of 
sight  of  the  HAH  the  participant  was  allowed  to  remove  the 
goggles. 

Exploration  of  the  virtual  environment.  Upon  return  to  ARI 
the  participant  was  taken  to  the  pilot  cockpit  of  STRATA,  seated 
and  given  a  brief  orientation  to  the  simulator.  The  participant 
was  shown  how  to  adjust  cockpit  ventilation  and  where  the 
emergency  shut-off  switches  were  located.  The  participant  was 
shown  the  FOHMD  and  how  to  handle  it  without  touching  the  optics 
The  participant  was  shown  the  two  joysticks  and  their  functions. 
Last,  each  participant  was  shown  the  air  sickness  bag.  Finally, 
the  participant  was  helped  into  the  FOHMD  and  a  communication 
check  was  made  with  the  experimenter  seated  at  the  EOS. 

All  participants  were  continuously  monitored  by  the 
experimenter  at  the  EOS.  Experimenter  and  participant  had  an 
open  microphone  link  so  that  during  exploration  of  the  VE  the 
experimenter  was  always  available  to  the  participant.  In 
addition,  the  experimenter  continuously  monitored  the  view  seen 
by  the  participant  in  the  helmet  via  the  repeaters  at  the  EOS. 
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All  participants  were  told  that  they  were  to  have  three 
exploration  sessions  of  thirty  minutes  each.  They  were 
instructed  to  learn  as  much  as  they  could  about  the  VE  because 
they  would  be  asked  questions  later.  They  were  reminded  that 
they  could  stop  for  a  break  at  any  time  if  they  wanted  a  rest,  if 
their  helmet  developed  a  "hot  spot,"  or  if  they  began  to  feel 
uncomfortable . 

Experience  in  the  VE  began  at  the  gasoline  station  reset 
position  in  the  Ariz-VE  for  all  participants  regardless  of  group. 
All  were  instructed  to  practice  controlling  their  virtual  carpet 
using  the  two  joysticks.  After  three  minutes  of  joystick  control 
practice  in  this  vicinity,  participants  were  told  to  return  to 
the  initial  location  by  pushing  the  reset  button. 

Up  until  now  the  procedure  was  the  same  for  members  of  both 
groups.  At  this  point  participants  who  were  members  of  the 
Control  Group  were  told  to  begin  their  first  thirty  minute 
exploration  session.  Participants  in  the  Hanchey  Group  were 
teleported  from  the  reset  point  in  the  Ariz-VE  to  the  reset  point 
in  the  HAH-VE  by  the  experimenter  at  the  EOS .  They  were  told  to 
begin  their  first  thirty  minute  exploration  session.  Members  of 
the  Control  Group  completed  three  thirty  minute  exploration 
sessions  in  the  Ariz-VE  and  members  of  the  Hanchey  Group 
completed  three  thirty  minute  exploration  sessions  in  the  HAH-VE. 
Sessions  were  separated  by  a  break  of  fifteen  minutes.  During 
the  break  each  participant  was  helped  off  with  the  FOHMD  and  got 
out  of  the  simulator  cockpit. 

Posttest.  Immediately  at  the  end  of  the  third  exploration 
session,  the  participant  completed  the  Posttest  Part  1 
Questionnaire.  This  questionnaire  was  identical  to  the  Pretest 
Part  l  Questionnaire  except  for  its  title.  Next  the  participant 
was  asked  to  fill  out  the  Posttest  Part  1  Object  Placement  Test. 
This  posttest  had  no  pretest  counterpart.  Both  of  these  posttest 
activities  took  place  at  ARI. 

Then  each  participant  was  again  taken  by  the  experimenter  to 
the  HAH.  The  participant  again  donned  opaque  goggles  prior  to 
getting  within  sight  of  Hanchey.  Upon  arrival  at  Hanchey,  the 
participant  was  lead  to  the  same  location  facing  the  Kiowa 
hangar.  This  was  also  the  same  location  as  the  reset  point  in 
the  HAH-VE.  Here  the  participant  was  once  again  instructed  to 
stand  still  with  head  and  eyes  fixed  forward.  With  the 
experimenter  positioned  in  front  of  the  participant,  the 
participant  removed  the  goggles  (which  were  not  used  again  that 
day) .  The  experimenter  read  the  instructions  for  the  Posttest 
Part  2  Questionnaire  (see  Appendix)  and  then  administered  it  to 
the  participant.  The  Posttest  Part  2  Questionnaire  was  identical 
to  the  Pretest  Part  2  Questionnaire  except  for  its  title.  The 
instructions,  administration,  and  scoring  procedures  were 
likewise  identical  to  those  already  described  for  the  pretest 
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counterpart. 

Up  to  this  point  the  posttest  procedures  were  the  same  for 
members  of  both  groups.  At  this  time  participants  in  the  Control 
Group  were  transported  back  to  ARI  to  complete  the  Presence 
Questionnaire.  Members  of  the  Hanchey  Group  provided  further 
information  as  to  their  knowledge  of  the  HAH. 

Members  of  the  Hanchey  Group  were  next  administered  the 
Hanchey  Army  Heliport  Walking  Navigation  Test.  This  transfer 
test  measured  participants'  knowledge  of  the  physical  features  of 
the  HAH  gained  in  the  virtual  environment  by  asking  them  to  use 
this  knowledge  to  navigate  from  one  location  to  another  at  the 
actual  HAH.  This  test  consisted  of  one  short  practice  walk  and 
two  walks  for  data  collection. 

The  experimenter  read  the  instructions  for  the  navigation 
test  (see  Appendix)  while  the  participant  remained  stationary 
facing  the  Kiowa  hangar.  The  practice  walk  required  the 
participant  to  walk  from  the  initial  position  at  the  southwest 
portion  of  the  Kiowa  hangar  to  the  nearest  field  elevation  sign 
which  was  located  on  the  southeast  portion  of  the  same  hangar. 

The  experimenter  followed  directly  behind  the  participant,  with 
stopwatch  and  clipboard,  recording  time  in  seconds,  number  of 
wrong  turns,  and  judged  confidence  level  of  the  participant.  The 
goal  position  (field  elevation  sign)  was  clearly  visible  from  the 
initial  position  and,  therefore,  this  walk  did  not  measure  the 
participant's  knowledge  of  HAH.  This  practice  walk  was  included 
to  confirm  that  the  participant  understood  the  procedure  and  to 
position  him/her  appropriately  for  the  data  collection  walk  to 
follow.  Experimental  data  were  not  collected.  None  of  the  goals 
or  waypoints  for  the  following  two  data  collection  walks  were 
visible  during  this  practice  walk. 

The  first  navigation  test  walk  required  the  participant  to 
walk  from  an  initial  position  under  the  field  elevation  sign  to 
the  two  silver  natural  gas  tanks,  passing  in  order  two  specific 
landmarks,  and  using  the  shortest  route  without  entering  any 
buildings.  Neither  the  goal  nor  the  waypoints  were  visible  from 
the  initial  position  or  from  each  other.  The  second  navigation 
test  walk  required  the  participant  to  walk  from  an  initial 
position  at  the  two  silver  natural  gas  tanks  to  the  front  of 
Cobra  Hall,  passing  in  order  two  specific  landmarks,  and  using 
the  shortest  route  without  entering  any  buildings.  Again, 
neither  the  goal  nor  the  waypoints  were  visible  from  the  initial 
position  or  from  each  other.  The  experimenter  followed  directly 
behind  the  participant  recording  the  three  measures  of 
performance  during  both  test  walks. 

This  navigation  test  was  not  given  to  members  of  the  Control 
Group  for  a  number  of  reasons.  First,  it  was  not  considered  safe 
to  have  a  demonstrably  naive  participant  wandering  around  a  busy, 
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working  heliport.  ARI  agreed  to  adhere  to  all  heliport  safety 
rules  upon  reguesting  access  to  Hanchey.  Second,  in  an  earlier 
practice  test  this  navigation  walk  was  found  to  be  an  impractical 
measure  of  performance  for  members  of  the  Control  Group.  In 
order  to  complete  the  task  an  uninformed  participant  must 
literally  search  until  he  or  she  has  found  the  goal  location  and 
both  waypoints  and  then  return  to  the  initial  position  and 
perform  the  task.  This  can  take  an  inordinate  amount  of  time  and 
produce  an  impossibly  large  number  of  wrong  turns.  The  HAH  is  a 
large,  complex,  detailed  area  to  be  searched  on  foot  under  the 
hot  Alabama  sun.  Third,  for  an  uninformed  member  of  the  Control 
Group  this  navigation  walk  becomes  a  search  task  rather  than  one 
of  navigation-from-memory .  This  is  a  different  task  than  the  one 
ostensibly  being  measured. 

Next,  the  members  of  the  Hanchey  Group  filled  out  the 
Hanchey  Army  Heliport  Knowledge  Rating  Form.  This  form  asked _ 
each  participant  to  rate  the  amount  of  knowledge  they  gained  in 
the  HAH-VE.  When  finished,  participants  were  transported  back  to 
ARI  to  complete  the  Presence  Questionnaire. 

Presence  Questionnaire,  debrief,  question  and  answer.  Upon 
their  return  to  ARI  all  participants  completed  the  Presence 
Questionnaire.  After  finishing  the  questionnaire  each 
participant  was  fully  debriefed  as  to  the  nature  of  the  research, 
the  experimental  design,  his/her  role  and  group  membership,  and 
any  questions  were  answered.  After  releasing  the  participant  the 
goggles  were  cleaned  with  isopropyl  alcohol  in  preparation  for 
the  next  day's  experiment. 


Results 


Scoring 

Objective  keys  and  reliable  scoring  procedures  were  created 
for  the  five  instruments  which  measured  knowledge  of  HAH.  These 
instruments  were:  Pretest  Part  1  Questionnaire,  Pretest  Part  2 
Questionnaire,  Posttest  Part  1  Questionnaire,  Posttest  Part  1 
Object  Placement  Test,  and  Posttest  Part  2  Questionnaire. 
Responses  recorded  on  these  instruments  were  scored  individually 
by  both  authors  (DJ,  DW)  .  Scorer  DJ,  being  the  experimenter,  was 
aware  of  which  group  (Hanchey  or  Control)  each  participant  was  a 
member  of  during  scoring.  Scorer  DW  was  blind  as  to  the  group 
membership  of  participants.  As  shown  in  Table  2  interscorer 
reliability  was  almost  perfect.  Scores  used  in  later  analyses 
were  those  of  DJ. 
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Table  2 


Interscorer  Reliability  of  Scorers  DJ  and  DW 


Instrument 

Correlation* 

Pretest 

Part  1 

Questionnaire 

r  = 

1.00, 

df  =  10, 

E<. 001 

Pretest 

Part  2 

Questionnaire 

r  = 

.99, 

df  =  10, 

p<. 001 

Posttest 

Part 

1  Questionnaire 

r  = 

1.00, 

df  =  10, 

£<. 001 

Posttest 

Part 

1  Object  Placement 

r  = 

.99, 

df  =  10, 

E<. 001 

Posttest 

Part 

2  Questionnaire 

r  = 

1.00, 

df  =  10, 

p<. 001 

*Pearson  Product-Moment  Correlation  Coefficient 


The  Presence  Questionnaire  was  scored  in  the  manner 
described  by  Witmer  and  Singer  (1994) .  Participants  placed  an 
"X"  in  one  of  seven  bins  along  a  seven-point,  Likert-type  scale. 
For  twenty-five  of  the  thirty-two  questions  the  left-most  bin  was 
scored  a  rating  of  1  and  the  right-most  bin  a  rating  of  7 .  Seven 
questions  were  reverse  scored,  with  the  left-most  bin  scored  as  a 

rating  of  7  and  the  right-most  bin  scored  as  a  rating  of  1.  The 

questions  which  were  reverse  scored  were  numbered  8,  9,  11,  24, 
25,  28,  and  29.  The  total  score  for  each  participant  was  the  sum 
of  the  ratings  from  all  thirty-two  questions.  The  Presence 
Questionnaire  was  scored  by  DJ. 

Analyses 

Knowledge  of  Hanchev  Army  Heliport.  The  purpose  of  this 

experiment  was  to  determine  if  knowledge  about  an  actual  location 

could  be  transmitted  via  VE  technology.  The  experiment  was  a 
simple  pretest-posttest  design  with  two  independent  groups.  The 
Hanchey  Group  explored  a  VE  representation  of  the  tested  location 
(HAH)  while  the  Control  Group  explored  a  VE  representation  of  a 
neutral  location  (Arizona) .  Therefore,  two  classes  of  planned, 
pairwise  comparisons  were  run.  First,  comparisons  between  the 
Hanchey  Group  and  Control  Group  on  all  pretest  and  posttest 
measures.  These  comparisons  employed  a  series  of  independent 
groups  t-tests.  All  pretest  comparisons  were  two-tailed  tests. 

No  significant  differences  in  either  direction  were  predicted. 

All  posttest  comparisons  were  one-tailed  tests.  Differences  were 
predicted  to  favor  the  Hanchey  Group.  Second,  comparisons  were 
made  between  pretest  performance  and  posttest  performance  for  all 
repeated  measures  for  both  groups.  These  comparisons  employed  a 
series  of  repeated  measures  t-tests.  All  pretest  versus  posttest 
comparisons  were  one-tailed  tests.  Differences  were  predicted  to 
favor  the  posttest. 
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Presence .  A  secondary  purpose  of  this  research  was  to 
measure  the  experience  of  presence  reported  by  participants  in 
the  VEs.  It  was  of  interest  to  learn  if  there  would  be  a 
difference  in  reported  presence  between  the  Hanchey  and  Control 
Groups.  The  two  groups  explored  VEs  which  differed  only  in 
content.  Therefore,  the  total  presence  score  was  compared 
between  groups.  This  planned,  pairwise  comparison  was  analyzed 
using  a  Mann-Whitney  U  test  in  addition  to  the  independent  groups 
t-test.  The  nonparametric  U  test  was  added  because  the  Presence 
Questionnaire  produced  ordinal-level  data  (but  see  Gaito,  1980) . 
Both  presence  comparisons  were  two-tailed  tests.  No  significant 
differences  in  either  direction  were  predicted. 

Knowledge  of  Hanchev  Army  Heliport 

Hanchey  Group  versus  Control  Group.  The  mean  scores  (and 
percent  correct)  of  the  Hanchey  Group  and  the  Control  Group  on 
the  five  instruments  measuring  knowledge  of  HAH  are  presented  in 
Table  3.  The  two  pretest  instruments  were  administered  to  all 
participants  prior  to  their  experience  in  the  VE.  The  three 
posttest  instruments  were  administered  to  all  participants  after 
their  experience  in  the  VE.  The  numbers  in  the  table  represent 
the  mean  and  standard  deviation  of  correct  answers  on  the 
measurement  instruments  by  group.  On  the  Pretest  Part  1 
Questionnaire  the  Hanchey  Group  mean  score  was  1.00  out  of  a 
possible  18,  while  that  for  the  Control  Group  was  0.50.  There 
was  no  statistically  significant  difference  between  the  groups  on 
this  questionnaire  (t(10)  =  0.70,  p>.10,  2-tailed).  On  the 
Pretest  Part  2  Questionnaire  both  groups  scored  1.00  out  of  a 
possible  22.  There  was  no  statistically  significant  difference 
between  the  groups  on  this  questionnaire  (t(10)  =  0.00,  p>.10,  2- 
tailed) .  The  results  showed  no  statistically  significant 
differences  between  the  groups  at  pretest  in  their  knowledge  of 
HAH. 


The  Hanchey  Group  scored  16.83  out  of  a  possible  18  on  the 
Posttest  Part  1  Questionnaire,  while  the  Control  Group  scored 
0.67.  This  difference  was  statistically  significant  (t(10)  = 
20.78,  pc. 001,  1-tailed).  On  the  Posttest  Part  1  Object 
Placement  Test  the  Hanchey  Group  scored  27.50  out  of  a  possible 
34,  while  the  Control  Group  scored  0.83.  This  difference  was 
statistically  significant  (t(10)  =  7.55,  pc.OOl,  1-tailed).  The 
Hanchey  Group  scored  20.67  out  of  a  possible  22  on  the  Posttest 
Part  2  Questionnaire,  while  the  Control  Group  scored  1.67.  This 
difference  was  statistically  significant  (t(10)  =  12.38,  p<.001, 
1-tailed) .  The  results  consistently  showed  statistically 
significant  differences  between  the  groups  at  posttest  in  their 
knowledge  of  HAH. 
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Table  3 


Comparison  of  Scores  (and  Percent  Correct)  of  Hanchey  Group  and 
Control  Group  on  Instruments  Measuring  Knowledge  of  Hanchey  Army 
Heliport  Before  and  After  Experience  in  the  Virtual  Environment 


Instrument 

Hanchey* 

Control* 

Pretest  Part  1  Questionnaire 

Mean 

SD 

1.00 

1.41 

(6%) 

0.50 

0.76 

(3%) 

Pretest  Part  2  Questionnaire 

Mean 

SD 

1.00 

1.41 

(5%) 

1.00 

1.41 

(5%) 

Posttest  Part  1  Questionnaire** 
Mean 

SD 

16.83 

1.34 

(94%) 

0.67 

1.11 

(4%) 

Posttest  Part  1  Object  Placement** 
Mean 

SD 

27.50 

7.76 

(81%) 

0.83 

1.46 

(2%) 

Posttest  Part  2  Questionnaire** 
Mean 

SD 

20.67 

2.98 

(94%) 

1.67 

1.70 

(8%) 

*  N  =  6  each  group 

**  Hanchey  Group  versus  Control  Group:  p<.001 


Pretest  versus  posttest.  The  Part  1  Questionnaire  and  Part 
2  Questionnaire  were  administered  both  before  exploration  of  the 
VE  (pretest)  and  after  this  exploration  (posttest) .  The  mean 
pretest  and  posttest  scores  (and  percent  correct)  on  these  two 
questionnaires  are  presented  in  Table  4  by  group.  The  Hanchey 
Group  scored  1.00  on  the  Part  1  Questionnaire  at  pretest  and 
16.83  at  pcsttest.  This  improvement  was  statistically 
significant  (t(5)  =  19.98,  p<.001,  1-tailed).  On  the  Part  2 
Questionnaire  the  Hanchey  Group  scored  1.00  at  pretest  and  20.67 
at  posttest.  This  improvement  was  also  statistically  significant 
(t (5)  =  13.33,  pc.001,  1-tailed).  The  results  showed 
statistically  significant  improvement  from  pretest  to  posttest  on 
both  repeated  measures  of  knowledge  for  the  Hanchey  Group. - 

The  Control  Group  showed  a  different  pattern  of  results.  On 
the  Part  1  Questionnaire  the  Control  Group  scored  0.50  at  pretest 
and  0.67  at  posttest  out  of  a  total  possible  score  of  18.  There 


31 


was  no  statistically  significant  difference  for  the  Control  Group 
between  pretest  and  posttest  performance  on  the  Part  1 
Questionnaire  (t(5)  =  1.00,  p>.10,  1-tailed).  On  the  Part  2 
Questionnaire  the  Control  Group  scored  1.00  at  pretest  and  1.67 
at  posttest  out  of  a  total  possible  score  of  22.  There  was  no 
statistically  significant  difference  between  pretest  and  posttest 
performance  for  the  Part  2  Questionnaire  (t(5)  =  2.00,  . 10>p>.05, 
1-tailed) .  The  results  consistently  showed  no  statistically 
significant  improvement  from  pretest  to  posttest  for  the  Control 
Group . 


Table  4 

Comparison  of  Pretest  versus  Posttest  Scores  (and  Percent 
Correct)  on  Instruments  Measuring  Knowledge  of  Hanchey  Army 
Heliport  for  Both  Groups 


Group* 

Pretest 

Posttest 

Hanchev 

Part  1  Questionnaire** 

Mean 

SD 

1.00 

1.41 

(6%) 

16.83 

1.34 

(94%) 

Part  2  Questionnaire** 

Mean 

SD 

1.00 

1.41 

(5%) 

20.67 

2.98 

(94%) 

Control 

Part  1  Questionnaire 

Mean 

SD 

0.50 

0.76 

(3%) 

0.67 

1.11 

(4%) 

Part  2  Questionnaire 

Mean 

SD 

1.00 

1.41 

(5%) 

1.67 

1.70 

(8%) 

N  =  6  each  group 
Pretest  versus  posttest:  p<.001 


Other  measures.  In  addition  to  the  instruments  reported 
above,  the  Hanchey  Group  provided  two  other  measures  of  their 
knowledge  of  HAH  during  the  posttest  phase  of  the  experiment. 
First,  they  participated  in  the  Hanchey  Army  Heliport  Walking 
Navigation  Test.  Then  they  rated  the  amount  of  knowledge  they 
thought  they  had  gained  in  their  exploration  of  the  HAH-VE  on  the 
Hanchey  Army  Heliport  Knowledge  Rating  Form. 


32 


The  Walking  Navigation  Test  measured  the  time  taken,  the 
number  of  wrong  turns  made,  and  the  judged  confidence  level  of 
each  participant  while  performing  two  self-guided  navigation 
trials  on  the  actual  HAH.  In  the  first  test  trial  the  mean  time 
taken  to  walk  from  initial  location  to  goal  location  was  315.50 
seconds.  No  participant  made  any  wrong  turns  (mean  =  0) .  All 
participants  were  judged  to  have  a  confidence  level  of  3  or 
"sure"  (mean  =  3) .  By  comparison,  the  experimenter  (DJ)  who 
designed  the  walking  test  and  was  intimately  familiar  with  the 
route  from  actual  experience  required  335  seconds  to  walk  the 
test  route  during  a  preexperimental  calibration  trial.  Four  of 
the  six  Hanchey  Group  participants  walked  the  route  in  less  time 
than  the  experimenter,  based  only  on  their  knowledge  of  HAH 
acquired  in  the  VE. 

In  the  second  test  trial  the  mean  time  taken  by  the  Hanchey 
Group  participants  was  475.33  seconds.  Again,  no  participant 
made  any  wrong  turns  (mean  =  0)  and  all  participants  were  judged 
to  have  a  confidence  level  of  3  or  "sure"  (mean  =  3) .  Again,  by 
comparison,  the  experimenter  who  designed  the  test  required  490 
seconds  to  walk  the  route  during  a  preexperimental  calibration 
trial.  Three  of  the  six  participants  completed  this  trial  route 
in  less  time  than  the  experimenter,  based  only  on  their  knowlege 
of  HAH  acquired  in  the  VE. 

All  members  of  the  Hanchey  Group  filled  out  the  HAH 
Knowledge  Rating  Form  which  required  them  to  rate  the  amount  of 
knowledge  they  had  gained  in  the  VE.  Five  of  the  six 
participants  rated  their  knowledge  gained  as  5  ("Very  Much")  and 
one  rated  his  at  4  ("Much")  for  a  mean  group  rating  of  4.83. 

Presence  Questionnaire 

All  participants  were  administered  the  Presence 
Questionnaire  in  order  to  assess  their  degree  of  psychological 
immersion  or  presence  in  the  virtual  environment  they  explored. 
Presence  Questionnaire  ratings  are  presented  in  Table  5  for  each 
group  separately  and  for  both  groups  combined.  There  was  no 
statistically  significant  difference  between  the  Hanchey  Group 
and  the  Control  Group  in  presence  ratings  (t(10)  =  1.08,  p>.10, 
2-tailed;  Mann-Whitney  U(6,6)  =  12,  p>.10,  2-tailed).  The  mean 
presence  rating  for  both  groups  combined  was  157.75. 
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Table  5 


Mean  Presence  Questionnaire  Ratings  for  Hanchey  Group,  Control 
Group,  and  Combined 


Mean 

SD 

N 


Hanchey  Group* 

154.83 

7.62 

6 


Control  Group* 

160.67 

9.37 

6 


Both  Groups  Combined 


Mean 

SD 

N 


157.75 

9.03 

12 


Hanchey  Group  versus  Control  Group:  p>.10 


Simulator  Sickness 


Because  simulator  sickness  was  not  a  research  issue  under 
investigation  in  this  experiment,  it  was  not  systematically . 
measured  for  all  participants.  Thirteen  soldiers  were  originally 
scheduled  for  participation  in  this  experiment.  Two  participants 
experienced  simulator  sickness  and  were  requested  to  fill  out  the 
Simulator  Sickness  Questionnaire  (SSQ)  of  Kennedy,  Lane,  Berbaum, 
and  Lilienthal  (1993)  .  One  of  the  two  participants  reported 
sickness  of  sufficient  severity  to  cause  him  to  request 
withdrawal  from  the  experiment.  This  participant  did  not  vomit 
but  did  report  on  the  SSQ  moderate  or  severe  symptoms  of: 
general  discomfort,  fatigue,  headache,  eyestrain,  nausea, 
fullness  of  head,  blurred  vision,  dizziness,  vertigo,  and  stomach 
awareness.  These  symptoms  all  occurred  within  the  first  fifteen 
minutes  of  experience  in  the  virtual  environment.  This 
participant  had  been  assigned  to  the  Hanchey  Group  and  was 
exploring  the  virtual  Hanchey.  As  for  predisposing  factors,  this 
participant  reported  many  fewer  hours  of  sleep  than  usual  the 
night  prior  to  the  experiment.  This  participant  was  excused  from 
the  experiment  and  his  data  were  not  included  in  the  results  of 
this  report. 

The  other  participant  vomited  into  his  air  sickness  bag 
eight  minutes  into  his  third  session  in  the  VE  (68  minutes  into 
the  90  minute  total  session  length) .  This  participant  had  not 
reported  any  symptoms  over  the  intercom  prior  to  his  sickness. 

He  expressed  surprise  that  the  sickness  had  come  upon  him  so 
suddenly.  He  reported  moderate  symptoms  of  dizziness  and  stomach 
awareness  on  the  SSQ.  This  participant  had  been  assigned  to  the 
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Control  Group  and  was  exploring  the  virtual  Arizona.  As  for 
predisposing  factors,  he  had  received  many  fewer  hours  of  sleep 
than  usual  the  three  nights  prior  to  the  experiment  because  of 
his  Army  duties.  This  participant  did  not  request  withdrawal 
from  the  experiment  and  in  fact  reported  he  felt  fine  after  the 
vomiting  episode.  After  a  25  minute  break  during  which  he  filled 
out  the  SSQ,  this  participant  returned  to  the  Ariz-VE  and 
finished  the  experiment.  His  data  were  included  among  those  in 
the  Control  Group. 


Discussion 

The  primary  purpose  of  this  experiment  was  to  determine  if 
VE  technology  can  be  used  to  familiarize  participants  with  a 
place  prior  to  their  going  there.  That  is,  can  important  terrain 
information  be  transmitted  via  this  medium  and  will  this 
information  transfer  to  the  actual  physical  location?  Another 
purpose  was  to  measure  the  presence  reported  by  participants  in 
each  of  the  two  virtual  environments — the  HAH-VE  for  the  Hanchey 
Group  and  the  Ariz-VE  for  the  Control  Group. 

Knowledge  of  Hanchev  Army  Heliport 

A  variety  of  converging  measurements  support  the  conclusion 
that  the  participants  in  this  experiment  were  able  to  learn 
terrain  knowledge  of  the  HAH  using  VE  technology  and  were  able  to 
transfer  this  information  to  the  actual  heliport.  One  of  the 
selection  criteria  for  this  experiment  was  that  participants  had 
never  visited  the  HAH.  All  participants  signed  a  form  stating 
that  they  had  not.  This  lack  of  knowledge  of  HAH  was  amply 
demonstrated  at  pretest  by  both  the  Hanchey  Group  and  the  Control 
Group  (see  Table  3) .  On  the  Pretest  Part  1  Questionnaire 
administered  at  ARI  both  groups  scored  near  zero  in  their 
knowledge  of  HAH  (6%,  3%).  On  the  Pretest  Part  2  Questionnaire 
administered  at  Hanchey  both  the  Hanchey  Group  and  the  Control 
Group  again  scored  near  zero  in  their  knowledge  of  HAH  (5%,  5%). 
There  were  no  statistically  significant  differences  between  the 
groups  in  their  knowledge  of  HAH  at  pretest. 

After  exploration  of  their  respective  virtual  environments 
the  situation  was  dramatically  different  (see  Table  3) .  The 
Control  Group,  which  had  been  exploring  the  neutral  Ariz-VE, 
continued  to  score  near  zero  on  both  the  Posttest  Part  1 
Questionnaire  (4%)  and  the  Posttest  Part  2  Questionnaire  (8%) . 

The  Control  Group  did  not  show  a  statistically  significant 
improvement  from  pretest  to  posttest  on  either  questionnaire  (see 
Table  4).  In  addition,  the  Control  Group  scored  near  zero  on  the 
Posttest  Part  1  Object  Placement  Test  (2%;  see  Table  3)  which  was 
a  posttest  measure  only. 
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The  Hanchey  Group,  however,  had  been  exploring  the  HAH-VE 
and  had  learned  a  good  deal.  At  posttest  the  Hanchey . Group 
scored  near  perfect  on  both  the  Posttest  Part  1  Questionnaire  and 
the  Posttest  Part  2  Questionnaire  (94%,  94%;  see  Table  3).  The 
Hanchey  Group  showed  statistically  significant  improvement  from 
pretest  to  posttest  on  both  questionnaires  (see  Table  4) .  In 
addition,  the  Hanchey  Group  scored  81  percent  correct  on  the 
Posttest  Part  1  Object  Placement  Test  (see  Table  3) .  At  posttest 
the  Hanchey  Group  scored  significantly  and  substantially  better 
than  the  Control  Group  on  all  three  measures  of  knowledge  where 
the  two  groups  were  compared  directly  (see  Table  3) . 

Clearly,  the  results  of  this  experiment  show  that 
participants  were  able  to  gain  knowledge  of  the  actual  HAH  from 
the  virtual  HAH.  On  tests  of  knowledge  administered  both  at  ARI 
and  at  HAH  participants  in  the  Hanchey  Group  went  from  scores 
near  zero  to  scores  near  perfect  after  ninety  minutes  of 
exploration  in  the  virtual  Hanchey.  On  these  same  tests  members 
of  the  Control  Group  scored  near  zero  at  pretest  and  showed  no 
improvement  at  posttest.  The  existence  of  the  Control  Group  in 
this  experiment  means  that  the  improvement  seen  by  the  Hanchey 
Group  cannot  have  been  caused  by  differences  at  pretest,  by 
repeated  knowledge  testing,  by  exploration  of  just  any  virtual 
environment,  or  by  the  mere  passage  of  time.  The  only 
parsimonious  explanation  of  these  results  is  that  the  Hanchey 
Group  learned  about  the  HAH  during  their  exploration  of  the 
virtual  Hanchey.  That  is,  external  terrain  knowledge  can  be 
transmitted  by  means  of  VE  technology. 

Not  only  were  the  Hanchey  Group  participants  able  to  learn 
about  Hanchey  from  the  virtual  environment  but  they  were  able  to 
transfer  this  knowledge  to  the  actual  site.  All  members  of  the 
Hanchey  Group  executed  two  trials  of  the  HAH  Walking  Navigation 
Test  immediately  after  the  questionnaire  portion  of  the  posttest. 
On  two  different  trials  of  this  test  participants  were  asked  to 
find  their  way  from  an  initial  starting  point  to  a  goal  point 
using  the  shortest  possible  route  without  passing  through 
buildings.  On  their  way  from  start  to  goal  they  were  to  pass  two 
landmarks  in  order.  Neither  the  goal  points  nor  the  en  route 
landmarks  were  visible  from  the  starting  points  or  from  each 
other.  Neither  the  goal  nor  the  waypoints  for  trial  number  two 
were  visible  during  trial  number  one.  Further,  the  correct  route 
on  each  trial  was  a  circuitous  one.  In  other  words,  this  task 
could  not  be  performed  quickly,  confidently,  and  without  error 
either  by  chance  or  by  visual  search.  To  perform  this  task 
perfectly — which  they  all  did — participants  were  required  to  have 
developed  some  sort  of  detailed  mental  representation  of  HAH 
while  exploring  the  VE  and  be  able  to  apply  this  knowledge  to 
navigating  the  actual  site. 
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The  results  showed  that  all  participants  exuded  confidence 
as  they  performed  both  trials  with  zero  wrong  turns  and  with  no 
wasted  time  spent  deciding  or  searching.  The  mean  times  to  reach 
the  goal  points  were  lower  for  the  members  of  the  Hanchey  Group 
than  were  those  times  recorded  for  the  experimenter  who  not  only 
designed  the  test  routes  but  who  was  intimately  familiar  with 
both  the  virtual  and  the  physical  HAH.  Some  participants  walked 
the  routes  so  rapidly,  in  fact,  that  the  (older)  experimenter- 
following  with  his  stopwatch  and  clipboard— had  to  admonish  them 
to  slow  down  so  he  could  keep  up.  The  participants  showed  no 
difficulty  whatsoever  transferring  the  knowledge  gained  in  the 
virtual  environment  to  the  navigation  task  in  the  actual 
environment. 

These  objective  measures  Of  performance  were  consistent  with 
the  self  reports  of  the  Hanchey  Group  participants.  At  the  end 
of  the  posttest,  while  still  at  HAH,  participants  were  given  an 
opportunity  to  rate  the  amount  they  learned  in  the  HAH-VE  on  a 
five-point  scale.  The  mean  self  rating  was  4.83  or  "Very  Much." 
The  anchoring  description  provided  for  this  rating  was:  "I 
learned  a  great  deal  in  the  virtual  environment.  I  feel  highly 
knowledgeable  about  Hanchey  now."  Consistent  with  their  posttest 
performance  participants  believed  they  had  learned  a  substantial 
amount  from  their  exploration  of  the  virtual  Hanchey. 

In  summary,  this  research  has  employed  three  types  of 
measures  to  investigate  whether  virtual  environment  technology 
can  be  used  to  provide  terrain  familiarization  training  to 
soldiers.  The  three  measurement  types  were:  questionnaire-based 
tests  of  knowledge  administered  both  at  the  actual  terrain 
location  and  at  a  remote  location;  performance-based  navigation 
tests  conducted  at  the  actual  terrain  location;  and  self  reports 
of  amount  learned.  All  measures  consistently  showed  high  levels 
of  learning.  There  does  not  appear  much  room  for  doubt  that 
terrain  information  can  be  transmitted  via  virtual  environment 
technology.  Whether  this  technology  is  efficient  or  cost- 
effective  relative  to  other  media  used  for  such  training  is  a 
subject  for  further  research. 

Presence 


All  participants  rated  the  amount  of  presence  they 
experienced  using  the  questionnaire  developed  by  Witmer  and 
Singer  (1994) .  There  was  no  statistically  significant  difference 
in  rated  presence  between  the  Hanchey  Group  and  the  Control  Group 
(see  Table  5) .  This  is  an  understandable  result  since  the  only 
difference  between  the  virtual  experiences  provided  the  two 
groups  was  the  content  of  the  virtual  environments.  The  Hanchey 
Group  explored  the  HAH-VE  while  the  Control  Group  explored  a 
section  of  the  Ariz-VE.  Even  these  different  locations  were 
similar  in  general  ways.  For  example,  both  environments 
contained  a  similar  density  of  artifacts  (buildings,  towers, 
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roads)  and  both  contained  the  visible,  virtual  compass  arrow. 

All  the  equipment  used  to  present  the  virtual  experience  was 
identical  for  both  groups.  The  helmet  and  headtracking 
technology  was  identical.  The  movement  control  via  left  and 
right  joysticks  was  identical.  The  sole  difference  was,  again, 
the  location  modeled  in  the  VE.  Under  such  stable  conditions  as 
these  one  would  be  surprised — even  suspicious — if  significant 
differences  in  rated  presence  did  emerge. 

The  mean  presence  score  in  this  experiment  of  157.75  (SD  = 
9.03,  N  =  12)  was  reasonably  consistent  with  other  reported 
scores  using  the  same  questionnaire,  once  differences  in 
experimental  method  were  taken  into  consideration.  Witmer, 
Bailey,  and  Knerr  (in  preparation)  reported  a  mean  presence  score 
of  147.25  (SD  =  15.20,  N  =  20).  Their  experiment  employed  a 
different  category  of  participants  (college  students)  and  a 
different  experimental  apparatus.  The  experimental  task, 
however,  was  similar.  Participants  used  the  VE  technology  to 
learn  a  complicated  route  through  the  interior  of  a  building  and 
then  transferred  this  knowledge  to  a  test  in  the  actual  building. 

Witmer  and  Singer  (1994)  reported  a  mean  presence  score  of 
144.29  (SD  =  16.68,  N  =  16)  for  the  first  administration  to  their 
sample  and  138.29  (SD  =  23.32,  N  =  16)  for  the  second 
administration.  Their  experiment  also  employed  college  students 
and  a  different  experimental  apparatus.  In  addition,  the  task 
was  quite  different.  Participants  performed  a  series  of 
perceptual  and  psychomotor  tasks  in  a  predetermined  order  within 
the  VE.  Given  these  differences  in  method,  differences  in  rated 
presence  would  not  be  unexpected. 

Research  to  determine  the  impact  of  different  participants, 
equipment,  and  procedures  on  rated  presence  will  require 
dedicated  experiments  which  manipulate  these  independent 
variables  directly  and  measure  presence  within  the  same  design. 
Given  the  variability  noted  above  in  the  presence  ratings  (see 
SDs)  within-subjects  rather  than  independent-groups  designs  would 
probably  be  preferable  wherever  possible. 


Conclusion 

Soldiers  were  able  to  learn  external  terrain  information 
from  self-guided  exploration  of  a  virtual  environment  and 
transfer  this  knowledge  to  the  actual,  physical  environment  that 
had  been  modeled.  The  existence  of  the  Control  Group  in  this 
experiment  rules  out  possible  alternative  explanations  for  the 
improvement  in  performance  seen  by  the  soldiers  trained  in  the 
virtual  environment.  Thus,  under  the  criteria  discussed  earlier 
in  this  report,  virtual  environment  technology  has  been  shown  to 
be  a  valid  medium  for  the  transmission  of  terrain  knowledge. 
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APPENDIX 


Forms,  questionnaires,  and  other  measures  of  performance 


A-l 


Date _  #_ _ 

FAMILIARIZATION  WITH  VIRTUAL  ENVIRONMENTS 
Demographic  Information  Form 

All  information  you  provide  will  be  used  for  research  purposes 
only.  Your  anonymity  is  assured. 

1.  Name:  _ _ _ _ _ 

Last  First  Middle 

2.  Social  Security  No.: _ - _ - _ _ 

3.  What  is  your  age?  _  Years 

4.  What  is  your  current  rank?  _ 

5.  To  which  unit  are  you  assigned?  _ _ 

6.  Daytime  Telephone  Number (s) :  __ _ 

7.  Are  you  an  aviator?  _ _ _ 

8.  Have  you  ever  visited  Hanchey  Army  Heliport?  _ 


Your  Signature 


A-3 


Date _ _  # _ 

Familiarization  with  Virtual  Environments 
Pretest  Part  1  Questionnaire 


Please  answer  all  questions.  Circle  the  correct  answer  where 
appropriate.  Guessing  is  permitted.  If  you  do  not  know  what  the 
answer  is  and  do  not  wish  to  guess,  write  "DK"  for  don't  know. 


1.  Which  helicopter  type  (types)  is  (are)  based  on  the  West  ramp 
at  Hanchey  Army  Heliport? 


2.  Which  helicopter  type  (types)  is  (are)  based  on  the  East  ramp 
at  Hanchey  Army  Heliport? 


3.  In  which  compass  orientation  are  the  helicopters  parked  on 
the  West  ramp  at  Hanchey?  North-South  orientation  or  East-West 
orientation? 


4.  In  which  compass  orientation  are  the  helicopters  parked  on 
the  East  ramp  at  Hanchey?  North-South  orientation  or  East-West 
orientation? 


5.  How  many  helipads  are  located  on  the  West  ramp  of  Hanchey? 


6.  How  are  these  helipads  identified  or  designated?  (That  is, 
list  the  identification  or  designation  of  each  of  these 
helipads. ) 


7.  Which  traffic  pattern  is  at  a  higher  altitude?  West  Hanchey 
traffic  pattern  or  East  Hanchey  traffic  pattern? 


8.  How  many  windsocks  are  located  at  Hanchey? 


A-5 


9.  Where  is  (are)  the  windsock(s)  located  relative  the  control 
tower?  Use  compass  direction  (i.e.,  North,  South,  East,  West)  to 
locate  each  windsock. 


10.  Where  is  the  beacon  tower  located  relative  to  the  control 
tower?  North,  South,  East,  or  West? 


11.  Where  is  the  fire  station  located  relative  to  the  control 
tower?  North,  South,  East,  or  West? 


12.  Where  are  the  fuel  tanks  located  relative  to  the  control 
tower?  North,  South,  East,  or  West? 


13 .  How  many  fuel  tanks  are  there? 


14.  What  is  the  field  elevation  of  Hanchey  in  feet? 


Date _ _  # _ 

Familiarization  with  Virtual  Environments 
Pretest  Part  2  Questionnaire 


I  am  going  to  remove  your  goggles.  Hold  your  head  and  eyes 
steady  and  pointed  forward.  Do  not  turn  your  head  to  the  left  or 
the  right.  Do  not  move  from  this  spot.  I  am  going  to  ask  you 
some  questions.  Please  answer  all  questions.  Guessing  is 
permitted.  If  you  do  not  know  an  answer  and  do  not  wish  to 
guess,  say  "don't  know." 


a.  Do  you  see  the  "Warrior  Country"  logo?  Yes  No 


1.  Where  is  the  control  tower  located  relative  to  your  position? 
To  your  front,  back,  left,  or  right?  DK 


2.  Where  is  the  fire  station  located  relative  to  your  position? 
To  your  front,  back,  left,  or  right?  DK 


3.  Which  helicopter  type  or  types  are  parked  to  your  left? 
AH- 6 4  Apache  AH-1  Cobra  CH-47  Chinook  OH-58  Kiowa 
UH-60  Blackhawk  UH-1  Huey  TH-67  Creek  RAH-66  Comanche 
DK 


4.  Which  helicopter  type  or  types  are  parked  to  your  right? 
AH-64  Apache  AH-1  Cobra  CH-47  Chinook  OH-58  Kiowa 
UH-60  Blackhawk  UH-1  Huey  TH-67  Creek  RAH-66  Comanche 
DK 


5.  Where  is  Cobra  Hall  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


6.  Where  is  the  beacon  tower  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


7.  Where  is  the  antenna  pole  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 
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8.  Where  are  the  fuel  tanks  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


9.  Where  is  the  water  tank  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


10.  Where  is  the  nearest  "field  elevation'*  sign  to  your 
position?  To  your  front,  back,  left,  or  right?  DK 

11.  Where  is  taxi  lane  Delta  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 

12.  Where  is  taxi  lane  Echo  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


13.  Which  aircraft  traffic  pattern  is  at  a  higher  altitude,  the 
traffic  pattern  to  your  left  or  the  traffic  pattern  to  your 
right?  Left  Right  DK 


14. 


Relative  to  your  position,  is  there  a  windsock: 

To  your  front? 

Yes 

No  DK 

To  your  back? 

Yes 

No  DK 

To  your  left? 

Yes 

No  DK 

To  your  right? 

Yes 

No  DK 

15.  Where  is  Windjammers  Chinook  Hall  relative  to  your  position? 
To  your  front,  back,  left,  or  right?  DK 


16.  Where  are  the  two  silver  natural  gas  tanks  relative  to  your 
position?  To  your  front,  back,  left,  or  right?  DK 


17.  Where  is  the  satellite  receiver  dish  relative  to  your 
position?  To  your  front,  back,  left,  or  right?  DK 
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Date _  # _ 

Familiarization  with  Virtual  Environments 
Posttest  Part  1  Questionnaire 


Please  answer  all  questions.  Circle  the  correct  answer  where 
appropriate.  Guessing  is  permitted.  If  you  do  not  know  what  the 
answer  is  and  do  not  wish  to  guess,  write  "DK"  for  don't  know. 


1.  Which  helicopter  type  (types)  is  (are)  based  on  the  West  ramp 
at  Hanchey  Army  Heliport? 


2.  Which  helicopter  type  (types)  is  (are)  based  on  the  East  ramp 
at  Hanchey  Army  Heliport? 


3.  In  which  compass  orientation  are  the  helicopters  parked  on 
the  West  ramp  at  Hanchey?  North-South  orientation  or  East-West 
orientation? 


4.  In  which  compass  orientation  are  the  helicopters  parked  on 
the  East  ramp  at  Hanchey?  North-South  orientation  or  East-West 
orientation? 


5.  How  many  helipads  are  located  on  the  West  ramp  of  Hanchey? 


6.  How  are  these  helipads  identified  or  designated?  (That  is, 
list  the  identification  or  designation  of  each  of  these 
helipads. ) 


7.  Which  traffic  pattern  is  at  a  higher  altitude?  West  Hanchey 
traffic  pattern  or  East  Hanchey  traffic  pattern? 


8.  How  many  windsocks  are  located  at  Hanchey? 
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9.  Where  is  (are)  the  windsock (s)  located  relative  the  control 
tower?  Use  compass  direction  (i.e.,  North,  South,  East,  West)  to 
locate  each  windsock. 


10.  Where  is  the  beacon  tower  located  relative  to  the  control 
tower?  North,  South,  East,  or  West? 


11.  Where  is  the  fire  station  located  relative  to  the  control 
tower?  North,  South,  East,  or  West? 


12.  Where  are  the  fuel  tanks  located  relative  to  the  control 
tower?  North,  South,  East,  or  West? 


13 .  How  many  fuel  tanks  are  there? 


14.  What  is  the  field  elevation  of  Hanchey  in  feet? 
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Date _  # _ 

Familiarization  with  Virtual  Environments 
Posttest  Part  1  Object  Placement  Test 


Use  the  generalized  diagram  of  Hanchey  Army  Heliport  provided. 
Place  the  following  objects  on  the  diagram  in  their  correct 
locations.  Guessing  is  permitted.  If  you  do  not  know  the  answer 
and  do  not  wish  to  guess,  leave  the  question  blank  and  go  on. 
Please  read  through  all  questions  first  before  answering  any. 


1.  Put  a  "WS"  wherever  a  windsock  is  located. 

2.  Put  a  "CT"  where  the  control  tower  is  located. 

3.  Put  a  "BT"  where  the  beacon  tower  is  located. 

4.  Label  each  helipad  with  its  appropriate  identifying 
designation. 

5.  Put  an  "FT"  where  the  fuel  tanks  are  located. 

6.  Put  a  "CH"  where  Cobra  Hall  is  located. 

7.  Put  a  "WCH"  where  the  Windjammers  Chinook  Hall  is  located. 

8.  Put  an  "HWC"  where  the  Warrior  Country  hangar  is  located. 

9.  Put  an  "H"  where  each  of  the  other  two  hangars  are  located. 

10.  Put  an  "FE"  where  each  of  the  two  field  elevation  signs  are 
located. 

11.  Put  an  "FS"  where  the  fire  station  is  located. 

12.  Put  the  appropriate  helicopter  types,  by  name  or 
alphanumeric  designation,  on  the  appropriate  sides  of  the  ramp. 
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Date _  # _ 

Familiarization  with  Virtual  Environments 
Posttest  Part  2  Questionnaire 


I  am  going  to  remove  your  goggles.  Hold  your  head  and  eyes 
steady  and  pointed  forward.  Do  not  turn  your  head  to  the  left  or 
the  right.  Do  not  move  from  this  spot.  I  am  going  to  ask  you 
some  questions.  Please  answer  all  questions.  Guessing  is 
permitted.  If  you  do  not  know  an  answer  and  do  not  wish  to 
guess,  say  "don't  know." 


a.  Do  you  see  the  "Warrior  Country"  logo?  Yes  No 

1.  Where  is  the  control  tower  located  relative  to  your  position? 
To  your  front,  back,  left,  or  right?  DK 

2.  Where  is  the  fire  station  located  relative  to  your  position? 
To  your  front,  back,  left,  or  right?  DK 


3.  Which  helicopter  type  or  types  are  parked  to  your  left? 
AH- 6 4  Apache  AH-1  Cobra  CH-47  Chinook  OH-58  Kiowa 
UH-60  Blackhawk  UH-1  Huey  TH-67  Creek  RAH-66  Comanche 
DK 


4.  Which  helicopter  type  or  types  are  parked  to  your  right? 
AH-64  Apache  AH-1  Cobra  CH-47  Chinook  OH-58  Kiowa 
UH-60  Blackhawk  UH-1  Huey  TH-67  Creek  RAH-66  Comanche 
DK 


5.  Where  is  Cobra  Hall  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


6.  Where  is  the  beacon  tower  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


7.  Where  is  the  antenna  pole  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 
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8.  Where  are  the  fuel  tanks  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 

9.  Where  is  the  water  tank  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


10.  Where  is  the  nearest  "field  elevation"  sign  to  your 
position?  To  your  front,  back,  left,  or  right?  DK 

11.  Where  is  taxi  lane  Delta  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


12.  Where  is  taxi  lane  Echo  relative  to  your  position?  To  your 
front,  back,  left,  or  right?  DK 


13.  Which  aircraft  traffic  pattern  is  at  a  higher  altitude,  the 
traffic  pattern  to  your  left  or  the  traffic  pattern  to  your 
right?  Left  Right  DK 


14.  Relative  to  your  position,  is  there  a  windsock: 

To  your  front?  Yes  No  DK 

To  your  back?  Yes  No  DK 

To  your  left?  Yes  No  DK 

To  your  right?  Yes  No  DK 

15.  Where  is  Windjammers  Chinook  Hall  relative  to  your  position? 
To  your  front,  back,  left,  or  right?  DK 

16.  Where  are  the  two  silver  natural  gas  tanks  relative  to  your 
position?  To  your  front,  back,  left,  or  right?  DK 


17.  Where  is  the  satellite  receiver  dish  relative  to  your 
position?  To  your  front,  back,  left,  or  right?  DK 


A-14 


Familiarization  with  Virtual  Environments 
Hanchey  Army  Heliport  Walking  Navigation  Test 


Date 


PRACTICE  WALK.  [Starting  Point:  Warrior  Country  hangar  reset 
position]  There  are  two  field  elevation  signs  on  this  heliport. 
Please  walk  to  and  stand  under  the  nearest  field  elevation  sign. 
Use  the  shortest  route  without  passing  through  any  buildings. 

Do  you  understand?  Yes  No 


Time _  Wrong  Turns _ 

Judged  Confidence  Level _ _ 

1  =  Unsure,  looking,  searching,  halting  steps 

2  =  In  between  1  and  3 

3  =  Sure,  confident,  not  searching,  direct  path 


WALK  1.  [Starting  Point:  Under  field  elevation  sign  on  Warrior 
Country  hangar]  Please  walk  to  the  two  silver  natural  gas  tanks, 
passing  on  your  way  first  the  two  blue  port-a-potties  then  the 
other  field  elevation  sign.  Use  the  shortest  route  without 
passing  through  any  buildings. 

Do  you  understand?  Yes  No 


Time _  Wrong  Turns _ 

Judged  Confidence  Level _ . 

1  =  Unsure,  looking,  searching,  halting  steps 

2  =  In  between  1  and  3 

3  =  Sure,  confident,  not  searching,  direct  path 


WALK  2.  [Starting  Point:  Two  silver  natural  gas  tanks]  Please 
walk  to  the  front  of  Cobra  Hall,  passing  on  your  way  first  the 
satellite  receiver  dish  and  then  the  entrance  to  Windjammers 
Chinook  Hall.  Use  the  shortest  route  without  passing  through  any 
buildings. 

Do  you  understand?  Yes  No 


Time _  Wrong  Turns _ 

Judged  Confidence  Level _ _ 

1  =  Unsure,  looking,  searching,  halting  steps 

2  =  In  between  1  and  3 

3  =  Sure,  confident,  not  searching,  direct  path 
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Date _  # _ 

Familiarization  with  virtual  Environments 
Hanchey  Army  Heliport  Knowledge  Rating  Form 


As  a  result  of  the  self-guided  exploration  you  performed  in  the 
virtual  Hanchey  environment,  how  would  you  rate  the  level  of 
knowledge  you  gained  about  Hanchey  Army  Heliport?  Please  circle 
a  number  from  1  to  5  below. 


1  2  3  4  5 

None  Some  Adequate  Much  Very  Much 


Where , 

1  =  None.  I  learned  nothing  about  Hanchey  in  the  virtual 
environment . 

3  =  Adequate.  I  learned  enough  in  the  virtual  environment 
to  find  my  way  around  Hanchey. 

5  =  Very  Much.  I  learned  a  great  deal  in  the  virtual 

environment.  I  feel  highly  knowledgeable 
about  Hanchey  now. 
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Date 


# 


PRESENCE  QUESTIONNAIRE 

Version  2.0,  Bob  Witmer  &  Michael  J.  Singer 

Characterize  your  experience  in  the  virtual  environment,  by 
marking  an  "X"  in  the  appropriate  box  of  the  7-point  scale,  in 
accordance  with  the  question  content  and  descriptive  labels. 
Please  consider  the  entire  scale  when  making  your  responses,  as 
the  intermediate  levels  may  apply.  Answer  the  questions 
independently  in  the  order  that  they  appear.  Do  not  skip 
questions  or  return  to  a  previous  question  to  change  your  answer. 
ANSWER  AIL  QUESTIONS  WITH  REGARD  TO  THE  VIRTUAL  ENVIRONMENT. 

1 .  How  much  were  you  able  to  control  events? 


NOT  AT  ALL  SOMEWHAT  COMPLETELY 


2 .  How  responsive  was  the  environment  to  actions  that  you 
initiated  (or  performed) ? 


NOT 

RESPONSIVE 

MODERATELY 

RESPONSIVE 

COMPLETELY 

RESPONSIVE 

3 .  How  natural  did 

i  1  1 

your  interactions  with 

1  1 

the  environment  seem? 

1  1  1 

EXTREMELY 

ARTIFICIAL 

BORDERLINE 

COMPLETELY 

NATURAL 

4 .  How  completely  were  all  of  your  senses 

iii  i  i 

engaged? 

1  1 

1 

NOT 

ENGAGED 

MILDLY 

ENGAGED 

COMPLETELY 

ENGAGED 

5.  Hew  much  did  the 
you? 

visual  aspects  of  the 

environment 

involve 

1  1  _  1 

1  1 

1  1 

1 

NOT  AT  ALL 

SOMEWHAT 

COMPLETELY 
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EXTREMELY 

ARTIFICIAL 


BORDERLINE 


COMPLETELY 

NATURAL 


8.  How  aware  were  you  of  events  occurring  in  the  real  world 
around  you? 


NOT  AWARE 
AT  ALL 


MILDLY 

AWARE 


VERY  AWARE 


9 .  How  aware  were  you  of  your  display  and  control  devices? 


NOT  AWARE 
AT  ALL 


MILDLY 

AWARE 


VERY  AWARE 


10.  How  compelling  was  your  sense  of  objects  moving  through 
space? 


NOT  AT  ALL- 


MODERATELY 

COMPELLING 


VERY 

COMPELLING 


11.  How  inconsistent  or  disconnected  was  the  information  coming 
from  your  various  senses? 


NOT  AT  ALL 
INCONSISTENT 


SOMEWHAT 

INCONSISTENT 


VERY 

INCONSISTENT 


12.  How  much  did  your  experiences  in  the  virtual  environment 
seem  consistent  with  your  real  world  experiences? 


I _ I 

NOT 

CONSISTENT 


MODERATELY 

CONSISTENT 


VERY 

CONSISTENT 
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13.  Were  you  able  to  anticipate  what  would  happen  next  in 
response  to  the  actions  that  you  performed? 

I _ I _ I _ I _ I _ I _ I _ I 

NOT  AT  ALL  SOMEWHAT  COMPLETELY 


14.  How  completely  were  you  able  to  actively  survey  or  search 
the  environment  using  vision? 

I _ I _ I _ l_ _ I _ I _ I _ I 

NOT  AT  ALL  SOMEWHAT  COMPLETELY 


15.  How  well  could  you  identify  sounds? 

I _ I _ I _ I _ I _ I _ I _ I 

NOT  AT  ALL  SOMEWHAT  COMPLETELY 


16.  How  well  could  you  localize  sounds? 


NOT  AT  ALL  SOMEWHAT  COMPLETELY 


17.  How  well  could  you  actively  survey  or  search  the  virtual 
environment  using  touch? 


NOT  AT  ALL 


SOMEWHAT 


COMPLETELY 


18.  How  compelling  was  your  sense  of  moving  around  inside  the 
virtual  environment? 


COMPELLING 


MODERATELY 

COMPELLING 


COMPELLING 


19.  How  closely  were  you  able  to  examine  objects? 


NOT  AT  ALL 


PRETTY 

CLOSELY 


VERY 

CLOSELY 


20.  How  well  could  you  examine  objects  from  multiple  viewpoints? 

• _ l_ _ I _  I _ I  I  I  I 

NOT  AT  ALL  SOMEWHAT  EXTENSIVELY 


A-21 


21.  How  well  could  you  move  or  manipulate  objects  in  the  virtual 
environment? 


I _ I 

NOT  AT  ALL' 


_ I 

SOMEWHAT 


_ I 

EXTENSIVELY 


22 .  To  what  degree  did  you  feel  confused  or  disoriented  at  the 
beginning  of  breaks  or  at  the  end  of  the  experimental  session? 

I _ I _ I _ l_ _ I _ I _ I _ I 

NOT  AT  ALL  MILDLY  VERY 

DISORIENTED  DISORIENTED 


23 .  How  involved  were  you  in  the  virtual  environment  experience? 

I _ I _ I _ I _ I _ I _ I _ I 

NOT  MILDLY  COMPLETELY 

INVOLVED  INVOLVED  ENGROSSED 


24.  How  distracting  was  the  control  mechanism? 


NOT  AT  ALL  MILDLY  VERY 

DISTRACTING  DISTRACTING 

25 .  How  much  delay  did  you  experience  between  your  actions  and 
expected  outcomes? 


NO  DELAYS 


MODERATE  LONG 

DELAYS  DELAYS 


26.  How  quickly  did  you  adjust  to  the  virtual  environment 
experience? 


NOT  AT  ALL  SLOWLY  LESS  THAN 

ONE  MINUTE 


27 .  How  proficient  in  moving  and  interacting  with  the  virtual 
environment  did  you  feel  at  the  end  of  the  experience? 


I _ I 

NOT 

PROFICIENT 


_ l_ 

REASONABLY 

PROFICIENT 


VERY 

PROFICIENT 
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28.  How  much  did  the  visual  display  quality  interfere  or 
distract  you  from  performing  assigned  tasks  or  required 
activities? 


NOT  AT  ALL 


INTERFERED  PREVENTED 

SOMEWHAT  PERFORMANCE 


29.  How  much  did  the  control  devices  interfere  with  the 
performance  of  assigned  tasks  or  with  other  activities? 


NOT  AT  ALL 


INTERFERED 

SOMEWHAT 


INTERFERED 

GREATLY 


30 .  How  well  could  you  concentrate  on  the  assigned  tasks  or 
required  activities  rather  than  on  the  mechanisms  used  to  perforin 
those  tasks  or  activities? 


NOT  AT  ALL  SOMEWHAT  COMPLETELY 


31*  Did  you  learn  new  techniques  that  enabled  you  to  improve 
your  performance? 


I _ I 

NO 

TECHNIQUES 

LEARNED 


I _ I 

LEARNED- 

SOME 

TECHNIQUES 


LEARNED 

MANY 

TECHNIQUES 


32.  Were  you  involved  in  the  experimental  task  to  the  extent 
that  you  lost  track  of  time? 

i _ i _ i _ i _ i 

NOT  AT  ALL  SOMEWHAT 


COMPLETELY 


